Can the Blobstore in GWT/GAE be used as a database? Or is a new Blobstore created each time I launch the application? I would like to store information without losing it when the application is closed. But I can't seem to find a way to name a Blobstore and then reference it by its ID. Thanks!
If all you want to do is store a string I'd still suggest using the datastore.
Here's the complete python source to an App Engine app that retrieves, modifies, and stores some text in the datastore:
from google.appengine.ext import webapp, db
from google.appengine.ext.webapp import util
class TextDoc(db.Model):
text = db.TextProperty(default="")
class MainHandler(webapp.RequestHandler):
def get(self):
my_text_doc = TextDoc.get_or_insert('my_text_doc')
my_text_doc.text += "Blah, blah, blah. "
my_text_doc.put()
self.response.out.write(my_text_doc.text)
def main():
application = webapp.WSGIApplication([('/', MainHandler)],
debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
If you're working in Java it would be more verbose, but similar.
Related
I have a Entity with ~50k rows in Google Cloud Datastore, the stand alone not GAE. I am starting development with GAE and would like to query this existing datastore without having to import it to GAE. I have been unable to find a way to connect to an existing datastore Kind.
Basic code altered from Hello World and other guides im trying to get working as a POC.
import webapp2
import json
import time
from google.appengine.ext import ndb
class Product(ndb.Model):
type = ndb.StringProperty()
#classmethod
def query_product(cls):
return ndb.gql("SELECT * FROM Product where name >= :a LIMIT 5 ")
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
query = Product.query_product()
self.response.write(query)
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
Returned Errors are
TypeError: Model Product has no property named 'name'
Seems obvious that its trying to use a GAE datastore with the kind Product instead of my existing Datastore with Product already defined, But I cant find how to make that connection.
There is only one Google Cloud Datastore. App Engine does not have a datastore of its own - it works with the same Google Cloud Datastore.
All entities in the Datastore are stored for a particular project. If you are trying to access data from a different project, you will not be able to see it without going through special authentication.
I'm not too certain what it is you're trying to accomplish when you say that you would like to query this existing datastore without having to import it to GAE. I'm guessing that you have project A with the datastore with 50k rows, and you're starting project B. And you want to access the project A datastore from project B. If this is the case, and if you're trying to access the datastore from a different project, then maybe this previous answer that mentions remote api can help you.
Below is working code. I was pretty close at the time I made this original post but the reason I was getting no data back was because I was running my App locally. As soon as I actually deployed my code to App Engine it pulled from Datastore no problem.
import webapp2
import json
import time
from google.appengine.datastore.datastore_query import Cursor
from google.appengine.ext import ndb
class Product(ndb.Model):
name = ndb.StringProperty()
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
query = ndb.gql("SELECT * FROM Product where name >= 'a' LIMIT 5 ")
output = query.fetch()
#query = Product.query(Product.name == 'zubo - pre-owned - nintendo ds')
#query = Product.query()
#output = query.fetch(10)
self.response.write(output)
app = webapp2.WSGIApplication([
('/', MainPage),
], debug=True)
I'm trying to access data that ODK has pushed into the datastore. The below code words fine when I query an entity that I created via Python, which was called "ProductSalesData". The entity name ODK has given it's data is "opendatakit.test1". When I update the data model to class opendatakit.test1(db.Model) it obviously bombs due to a sytax error. How do I call that data?
#!/usr/bin/env python
import webapp2
from google.appengine.ext import db
class ProductSalesData(db.Model):
product_id = db.IntegerProperty()
date = db.DateTimeProperty()
store = db.StringProperty()
q = ProductSalesData.all()
class simplequery(webapp2.RequestHandler):
def get(self):
for ProductSalesData in q:
self.response.out.write('Result:%s<br />' % ProductSalesData.store)
app = webapp2.WSGIApplication(
[('/', simplequery)],
debug=True)
I know you tagged GAE, but do you have to access it straight through the datastore?
If not, I've had better success using the API that has already been built into aggregate: https://code.google.com/p/opendatakit/wiki/BriefcaseAggregateAPI
If you need GAE access I'd suggest the ODK developers group over on google groups - they're pretty active.
I am running a bottle webservice in localhost in my system. I have connected it to the MySQL and it is working fine, but when it comes to googleappengine how can i connect to NoSQL.
This is my code:
import bottle
import webapp2
from bottle import route, run, template
import MySQLdb, MySQLdb.cursors
from bottle import route, template, request ,debug
from google.appengine.ext.webapp.util import run_wsgi_app
db = MySQLdb.connect(user="root", passwd="root", db="delhipoc" , host='127.0.0.1')
cur = db.cursor()
def main():
debug(True)
run_wsgi_app(bottle.default_app())
#route('/hai/<name>')
def show(name):
print "the name is \n",name
cur.execute('SELECT * from poc_people')
print "i am in show \n"
for row in cur.fetchall() :
print row[0]
data = row[0]
print "hai \n",row[0]
return template('<b></br></br></br>Hello{{name}}</br></br></br>{{data}} </b>!',name=name,data=data)
if __name__ == '__main__':
main()
In the above way i am able to connect to my local mysql database. But how can i connect it to GAE database
Create a datastore model corresponding to your table and convert all mysql api request to use datastore api.
Datastore reference: https://developers.google.com/appengine/docs/python/datastore/
You don't need to explicitly "connect" to gae datastore, you can just issue the query, filter, etc.
Context:
My model classes inherit from a base class:
class BaseModel(ndb.model):
# commom fields and methods
class SpecificModel(BaseModel):
# specific fields and methods
Problem:
I want to export the SpecificModel entities using the appengine bulkuploader service.
I have the defined the config file (data_loader.py):
import sys
sys.path.append('.') ## this is to ensure that it finds the file 'models.py'
from google.appengine.ext import ndb
from google.appengine.tools import bulkloader
from models import *
class SpecificModelExporter(bulkloader.Exporter):
def __init__(self):
bulkloader.Exporter.__init__(self, 'SpecificModel',
[('fieldOne', str, None),
('fieldTwo', str, None)
])
exporters = [ SpecificModelExporter ]
I use the following command to download data:
appcfg.py download_data --config_file=data_loader.py --filename=data.csv --kind=SpecificModel --url=http://url.appspot.com/_ah/remote_api
When I try to download the data I get the following error:
google.appengine.ext.db.KindError: No implementation for kind 'SpecificModel'
Any clues?
Have a look at the source code:
Your model will be looked up in GetImplementationClass via
implementation_class = db.class_for_kind(kind_or_class_key)
but the registry of db models will not include any ndb models you've defined. A similar registry is created in ndb.Model._kind_map and any db models you had defined would not be found there.
NOTE: As far as I can tell there is no corresponding issue/feature request asking for ndb support in the bulk loader or an equivalent ndb bulk loader. It may be worth filing one and starring it.
My problem revolves around a user making a text file upload to my app. I need to get this file and process it with my app before saving it to the datastore. From the little I have read, I understand that user uploads go directly to the datastore as blobs, which is ok if I could then get that file, perform operations on it(meaning change data inside) and then re-write it back to the datastore. All these operations need to be done by the app.
Unfortunately from the datastore documenation, http://code.google.com/appengine/docs/python/blobstore/overview.html
an app cannot directly create a blob in the datastore. That's my main headache. I simply need a way of creating a new blob/file in the datastore from my app without any user upload interaction.
blobstore != datastore.
You can read and write data to the datastore as much as you like so long as your data is <1MB using a db.BlobProperty on your entity.
As Wooble comments, the new File API lets you write to the blobstore, but unless you are incrementally writting to the blobstore-file using tasks or something like the mapreduce library you are still limited by the 1MB API call limit for reading/writing.
Thanks for your help. After many sleepless nights, 3 App Engine Books and A LOT of Googling, I've found the answer. Here is the code (it should be pretty self explanatory):
from __future__ import with_statement
from google.appengine.api import files
from google.appengine.ext import blobstore
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write('Hello WOrld')
form=''' <form action="/" method="POST" enctype="multipart/form-data">
Upload File:<input type="file" name="file"><br/>
<input type="submit"></form>'''
self.response.out.write(form)
blob_key="w0MC_7MnZ6DyZFvGjgdgrg=="
blob_info=blobstore.BlobInfo.get(blob_key)
start=0
end=blobstore.MAX_BLOB_FETCH_SIZE-1
read_content=blobstore.fetch_data(blob_key, start, end)
self.response.out.write(read_content)
def post(self):
self.response.out.write('Posting...')
content=self.request.get('file')
#self.response.out.write(content)
#print content
file_name=files.blobstore.create(mime_type='application/octet-stream')
with files.open(file_name, 'a') as f:
f.write(content)
files.finalize(file_name)
blob_key=files.blobstore.get_blob_key(file_name)
print "Blob Key="
print blob_key
def main():
application=webapp.WSGIApplication([('/', MainHandler)],debug=True)
util.run_wsgi_app(application)
if __name__=='__main__':
main()