I'm trying to access data that ODK has pushed into the datastore. The below code words fine when I query an entity that I created via Python, which was called "ProductSalesData". The entity name ODK has given it's data is "opendatakit.test1". When I update the data model to class opendatakit.test1(db.Model) it obviously bombs due to a sytax error. How do I call that data?
#!/usr/bin/env python
import webapp2
from google.appengine.ext import db
class ProductSalesData(db.Model):
product_id = db.IntegerProperty()
date = db.DateTimeProperty()
store = db.StringProperty()
q = ProductSalesData.all()
class simplequery(webapp2.RequestHandler):
def get(self):
for ProductSalesData in q:
self.response.out.write('Result:%s<br />' % ProductSalesData.store)
app = webapp2.WSGIApplication(
[('/', simplequery)],
debug=True)
I know you tagged GAE, but do you have to access it straight through the datastore?
If not, I've had better success using the API that has already been built into aggregate: https://code.google.com/p/opendatakit/wiki/BriefcaseAggregateAPI
If you need GAE access I'd suggest the ODK developers group over on google groups - they're pretty active.
Related
I have a Entity with ~50k rows in Google Cloud Datastore, the stand alone not GAE. I am starting development with GAE and would like to query this existing datastore without having to import it to GAE. I have been unable to find a way to connect to an existing datastore Kind.
Basic code altered from Hello World and other guides im trying to get working as a POC.
import webapp2
import json
import time
from google.appengine.ext import ndb
class Product(ndb.Model):
type = ndb.StringProperty()
#classmethod
def query_product(cls):
return ndb.gql("SELECT * FROM Product where name >= :a LIMIT 5 ")
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
query = Product.query_product()
self.response.write(query)
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
Returned Errors are
TypeError: Model Product has no property named 'name'
Seems obvious that its trying to use a GAE datastore with the kind Product instead of my existing Datastore with Product already defined, But I cant find how to make that connection.
There is only one Google Cloud Datastore. App Engine does not have a datastore of its own - it works with the same Google Cloud Datastore.
All entities in the Datastore are stored for a particular project. If you are trying to access data from a different project, you will not be able to see it without going through special authentication.
I'm not too certain what it is you're trying to accomplish when you say that you would like to query this existing datastore without having to import it to GAE. I'm guessing that you have project A with the datastore with 50k rows, and you're starting project B. And you want to access the project A datastore from project B. If this is the case, and if you're trying to access the datastore from a different project, then maybe this previous answer that mentions remote api can help you.
Below is working code. I was pretty close at the time I made this original post but the reason I was getting no data back was because I was running my App locally. As soon as I actually deployed my code to App Engine it pulled from Datastore no problem.
import webapp2
import json
import time
from google.appengine.datastore.datastore_query import Cursor
from google.appengine.ext import ndb
class Product(ndb.Model):
name = ndb.StringProperty()
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
query = ndb.gql("SELECT * FROM Product where name >= 'a' LIMIT 5 ")
output = query.fetch()
#query = Product.query(Product.name == 'zubo - pre-owned - nintendo ds')
#query = Product.query()
#output = query.fetch(10)
self.response.write(output)
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
Context:
My model classes inherit from a base class:
class BaseModel(ndb.model):
# commom fields and methods
class SpecificModel(BaseModel):
# specific fields and methods
Problem:
I want to export the SpecificModel entities using the appengine bulkuploader service.
I have the defined the config file (data_loader.py):
import sys
sys.path.append('.') ## this is to ensure that it finds the file 'models.py'
from google.appengine.ext import ndb
from google.appengine.tools import bulkloader
from models import *
class SpecificModelExporter(bulkloader.Exporter):
def __init__(self):
bulkloader.Exporter.__init__(self, 'SpecificModel',
[('fieldOne', str, None),
('fieldTwo', str, None)
])
exporters = [ SpecificModelExporter ]
I use the following command to download data:
appcfg.py download_data --config_file=data_loader.py --filename=data.csv --kind=SpecificModel --url=http://url.appspot.com/_ah/remote_api
When I try to download the data I get the following error:
google.appengine.ext.db.KindError: No implementation for kind 'SpecificModel'
Any clues?
Have a look at the source code:
Your model will be looked up in GetImplementationClass via
implementation_class = db.class_for_kind(kind_or_class_key)
but the registry of db models will not include any ndb models you've defined. A similar registry is created in ndb.Model._kind_map and any db models you had defined would not be found there.
NOTE: As far as I can tell there is no corresponding issue/feature request asking for ndb support in the bulk loader or an equivalent ndb bulk loader. It may be worth filing one and starring it.
Tech Stack: ndb models, WTForms, webapp2
I was experimenting with the WTForms extension to appengine db models.
I had a simple db schema:
class Autho(ndb.Models):
name = db.StringProperty()
class Notes(ndb.Model):
title = db.StringProperty()
author = db.KeyProperty()
and then a simple form definition form WTForms, in the handlers as per their documentation:
from wtforms.ext.appengine.db import model_form
def get(self, slug):
form = model_form(Author)()
self.render_template('form.html', {'form': form})
This raises an Attribute Error line 411 here props = model.properties()
Please let me know, if this can be fixed.
I've never used WTForms, but it seems to be incompatible with NDB -- "model.properties()" is an old db idiom. Maybe the WTForms author will consider adding support for NBD?
Check this out: https://github.com/coto/gae-boilerplate.
It's a ndb boilerplate with a basic (not so basic for me) implementation of user authentication that accepts federated login. It uses WTForms thoroughly. The guys that created it are very well reachable and the boilerplate is constantly maintained and updated. Worth a look, it helps a lot with basic patterns.
Can the Blobstore in GWT/GAE be used as a database? Or is a new Blobstore created each time I launch the application? I would like to store information without losing it when the application is closed. But I can't seem to find a way to name a Blobstore and then reference it by its ID. Thanks!
If all you want to do is store a string I'd still suggest using the datastore.
Here's the complete python source to an App Engine app that retrieves, modifies, and stores some text in the datastore:
from google.appengine.ext import webapp, db
from google.appengine.ext.webapp import util
class TextDoc(db.Model):
text = db.TextProperty(default="")
class MainHandler(webapp.RequestHandler):
def get(self):
my_text_doc = TextDoc.get_or_insert('my_text_doc')
my_text_doc.text += "Blah, blah, blah. "
my_text_doc.put()
self.response.out.write(my_text_doc.text)
def main():
application = webapp.WSGIApplication([('/', MainHandler)],
debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
If you're working in Java it would be more verbose, but similar.
In the development admin console, when I look at my data, it says "Select different namespace".
What are namespaces for and how should I use them?
Namespaces allow you to implement segregation of data for multi-tenant applications. The official documentation links to some sample projects to give you an idea how it might be used.
Namespaces is used in google app engine to create Multitenant Applications. In Multitenent applications single instance of the application runs on a server, serving multiple client organizations (tenants). With this, an application can be designed to virtually partition its data and configuration (business logic), and each client organization works with a customized virtual application instance..you can easily partition data across tenants simply by specifying a unique namespace string for each tenant.
Other Uses of namespace:
Compartmentalizing user information
Separating admin data from application data
Creating separate datastore instances for testing and production
Running multiple apps on a single app engine instance
For More information visit the below links:
http://www.javacodegeeks.com/2011/12/multitenancy-in-google-appengine-gae.html
https://developers.google.com/appengine/docs/java/multitenancy/
http://java.dzone.com/articles/multitenancy-google-appengine
http://www.sitepoint.com/multitenancy-and-google-app-engine-gae-java/
Looking, towards this question is not that much good reviewed and answered so trying to give this one.
When using namespaces, we can have a best practice of key and value separation there on a given namespace. Following is the best example of giving the namespace information thoroughly.
from google.appengine.api import namespace_manager
from google.appengine.ext import db
from google.appengine.ext import webapp
class Counter(db.Model):
"""Model for containing a count."""
count = db.IntegerProperty()
def update_counter(name):
"""Increment the named counter by 1."""
def _update_counter(name):
counter = Counter.get_by_key_name(name)
if counter is None:
counter = Counter(key_name=name);
counter.count = 1
else:
counter.count = counter.count + 1
counter.put()
# Update counter in a transaction.
db.run_in_transaction(_update_counter, name)
class SomeRequest(webapp.RequestHandler):
"""Perform synchronous requests to update counter."""
def get(self):
update_counter('SomeRequest')
# try/finally pattern to temporarily set the namespace.
# Save the current namespace.
namespace = namespace_manager.get_namespace()
try:
namespace_manager.set_namespace('-global-')
update_counter('SomeRequest')
finally:
# Restore the saved namespace.
namespace_manager.set_namespace(namespace)
self.response.out.write('<html><body><p>Updated counters')
self.response.out.write('</p></body></html>')