No longer able to create an entity using the dev_appserver admin console - google-app-engine

I cleared all entities from the datastore and cleared the memcache, created an entity using the API Explorer and then attempted to create another one of the same kind in the Datastore console running on localhost:
Internal Server Error
The server has either erred or is incapable of performing the requested operation.
Traceback (most recent call last):
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1536, in __call__
rv = self.handle_exception(request, response, e)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/admin_request_handler.py", line 97, in dispatch
super(AdminRequestHandler, self).dispatch()
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py", line 760, in get
count=20)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py", line 121, in _get_entities
query = datastore.Query(kind, _namespace=namespace)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/api/datastore.py", line 1361, in __init__
datastore_errors.BadArgumentError)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/api/datastore_types.py", line 181, in ValidateString
raise exception('%s must not be empty.' % name)
BadArgumentError: kind must not be empty.
Here's the gcloud version info:
$ gcloud --version
Google Cloud SDK 157.0.0
app-engine-python 1.9.54
beta 2017.03.24
bq 2.0.24
core 2017.05.30
gcloud
gsutil 4.26
Any hints as to what might be wrong?

I think it's a bug in the Datastore console.
I was able to replicate the same error on my machine. If you look at the traceback in the exception, you see (fourth from the bottom):
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py", line 760, in get
count=20)
If you look at that file from line 735, you see:
def get(self, entity_key_string=None):
super(DatastoreEditRequestHandler, self).get(entity_key_string)
if entity_key_string:
entity_key = datastore.Key(entity_key_string)
entity_key_name = entity_key.name()
entity_key_id = entity_key.id()
namespace = entity_key.namespace()
kind = entity_key.kind()
entities = [datastore.Get(entity_key)]
parent_key = entity_key.parent()
if parent_key:
parent_key_string = _format_datastore_key(parent_key)
else:
parent_key_string = None
else:
entity_key = None
entity_key_string = None
entity_key_name = None
entity_key_id = None
namespace = self.request.get('namespace')
kind = self.request.get('kind')
entities, _ = _get_entities(kind,
namespace,
order=None,
start=0,
count=20) ### line 760 #######
...
In that last line, kind is supposed to be string containing the model name of the model you are working with. Instead, if you follow the traceback down into that function call, you see that kind is an empty string. I don't think there's much you can do to fix it at this time. I'd recommend using the interactive console for now to create/edit your Datastore entities, and just use the viewer to view them. I'm sure Google will fix this with an update soon.
Edit: I actually found a hacky fix if you really want the Datastore console to work, but I would still recommend just using the interactive console. The hacky fix is you manually enter the kind variable into the URL sent by the Datastore console.
When you click "Create New Entity" and get the error you show in your question, look at the url:
http://localhost:8000/datastore/edit?kind=&next=http%3A%2F%2Flocalhost%3A8000%2Fdatastore%3Fkind%3DUnique
You see ...:8000/datastore/edit?kind=&.... Right there, just type in the model name. For example, let's say your model name is User:
http://localhost:8000/datastore/edit?kind=User&next=http%3A%2F%2Flocalhost%3A8000%2Fdatastore%3Fkind%3DUnique
This brought me to the "Create new entity" form for class User (one of my models), and I was able to create a user with that form.
P.S. You may want to also change the model name on the very end of that url (where mine says Unique) to match the kind you are creating, because that model name is the kind that will be displayed after you create your new entity. So mine created a User, but then displayed the Unique entities (Unique is one of my model names).

Related

pyVmomi exception on manager.CreateImportSpec

I am trying to deploy ovf using pyVmomi. I am trying to follow this example: https://github.com/vmware/pyvmomi-community-samples/blob/master/samples/deploy_ovf.py.
There are two VCenter setups and there are differences. On one VCenter, I can deploy the VM. On the other, I have a failure as below:
>>> spec_result = manager.CreateImportSpec(ovfd, rp_obj, datastore_obj,spec_params)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/VmomiSupport.py", line 570, in <lambda>
self.f(*(self.args + (obj,) + args), **kwargs)
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/VmomiSupport.py", line 376, in _InvokeMethod
return self._stub.InvokeMethod(self, info, args)
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/SoapAdapter.py", line 1350, in InvokeMethod
raise obj # pylint: disable-msg=E0702
pyVmomi.VmomiSupport.InvalidType: (vmodl.fault.InvalidType) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = '',
faultCause = <unset>,
faultMessage = (vmodl.LocalizableMessage) [],
argument = 'ManagedObjectReference'
}
Can someone point to the way on how I can troubleshoot this error? I tried to look at what this function does but couldn't find it.
I tried to set pdb trace, and it turns out that the reason it fails is that this comes back with a 500 response. On the VCenter that worked, it returns a 200 response. So why would the same call return differently?
I used the wrong datastore. The one I used was not of type "Datastore":
>>> type(datastore_obj)
<class 'pyVmomi.VmomiSupport.vim.StoragePod'>
When I used one of its childEntity, it worked:
>>> type(datastore_obj.childEntity[0])
<class 'pyVmomi.VmomiSupport.vim.Datastore'>
Here is what helped with debugging: I put a breakpoint in SoapAdapter.py. When 500 is returned to me, it entered pdb, and I obtained my own request, which contained this line at the bottom:
<datastore type="StoragePod">datastore-1</datastore>
When I compared this with the other VCenter that worked, I saw this difference:
<datastore type="Datastore">datastore-10</datastore>

Attribute error while exporting data from app engine using Remote APi

Use the example code from app engine will give an attribute error. The more strange thing is,
When the batch_size is 100, the first fetch will give an error while if it were set to 10, the second fetch will give the error, when the batch_size is 1, the 25th fetch will give the error. Is it due to the problem of remote API?
Python version: 2.7
App engine sdk version: 1.9.6
query = MyModel.all()
entities = query.fetch(100)
while entities:
for entity in entities:
# Do something with entity
query.with_cursor(query.cursor())
entities = query.fetch(100)
error message:
Traceback (most recent call last):
File "migrate.py", line 77, in <module>
entities = query.fetch(batch_size)
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 2157, in fetch
return list(self.run(limit=limit, offset=offset, **kwargs))
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 2326, in next
return self.__model_class.from_entity(self.__iterator.next())
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 1435, in from_entity
entity_values = cls._load_entity_values(entity)
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 1413, in _load_entity_values
value = prop.make_value_from_datastore(value)
File "/home/kamel/labola/src/model/properties.py", line 295, in make_value_from_datastore
return pickle.loads(value)
File "/usr/lib/python2.7/pickle.py", line 1382, in loads
return Unpickler(file).load()
File "/usr/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1083, in load_newobj
obj = cls.__new__(cls, *args)
AttributeError: class Reference has no attribute '__new__
I encountered the same issue when trying to unpickle python3 pickles under python2. The problem was linked to new-style classes becoming default in python3. (source)
Solution for me was to replace class AClass: by class AClass(object):

Images from Google Cloud Storage no longer display after GAE 1.8.8 update

After updating to GAE 1.8.8 and the GCS Client to r127 my code that displays images no longer works using the Development Server or Production. Code that worked previously now throughs the following error in development:
Blob with key 'encoded_gs_file:blahblahblah' does not exist
Note blahblahblah is just my shorthand for a long string.
Below is how I generate my url. This has always worked in the past.
bkey = blobstore.create_gs_key('/gs/mybucket/myfile.jpg')
url = images.get_serving_url(bkey, size=170, crop=True, secure_url=True)
Below is a stack trace from the development server:
ERROR 2013-11-21 20:35:31,767 images_stub.py:405] Blob with key 'encoded_gs_file:c2l0ZS1hdWRpdC1waW5uYWNsZWNyZS9CYXR0ZXJ5L3Bob3Rvcy8yMDEzMTEyMTAwMTEyOUBhNzllYmQwNC05YWY3LTRhMWItYTg3My1mZGIwYjI5MjNjNzguanBn' does not exist
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/images/images_stub.py", line 401, in _OpenBlob
blobstore_stub.BlobstoreServiceStub.ToDatastoreBlobKey(blob_key))
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 651, in Get
return GetAsync(keys, **kwargs).get_result()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 612, in get_result
return self.__get_result_hook(self)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1541, in __get_hook
entities = extra_hook(entities)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 620, in local_extra_hook
raise datastore_errors.EntityNotFoundError()
EntityNotFoundError
ApplicationError(1,)
Reading the release notes for GAE 1.8.8 I cannot see anything else I need to change in my code. Any pointers as to what I am doing wrong would be great.

Migrating from Python 2.5 to 2.7 with appengine and django non-rel

I read the "what is a metaclass in Python" but am still confused over it.
I am new to python and have been thrown into upgrading it from 2.5 to 2.7.
I have the following:
class UsersDB(db.Model):
Email = db.EmailProperty(required=True,verbose_name='Email *')
Enable = db.BooleanProperty(default=True)
and
class UsersQuickAddForm(forms.ModelForm):
def is_user_exist(self, account):
users_query = UsersDB.all().filter('Email =', account).fetch(1)
if len(users_query) > 0:
return True
return False
class Meta:
model = UsersDB
exclude = ['Enable']
but when I try to execute it on the google site, I get:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 239, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 298, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 84, in LoadObject
obj = import(path[0])
File "/base/data/home/apps/s~ldsdgidev/glen27.371429613087607751/LDSGH.py", line 8, in
from core.decorators import permissionRequired
File "/base/data/home/apps/s~ldsdgidev/glen27.371429613087607751/core/decorators.py", line 7, in
from core.initialization import loginIf
File "/base/data/home/apps/s~ldsdgidev/glen27.371429613087607751/core/initialization.py", line 6, in
import photo_images
File "/base/data/home/apps/s~ldsdgidev/glen27.371429613087607751/core/photo_images.py", line 1, in
from core.db_models import ImagesDB
File "/base/data/home/apps/s~ldsdgidev/glen27.371429613087607751/core/db_models.py", line 222, in
class UsersQuickAddForm(forms.ModelForm):#only account, firstname and last name is required
File "/base/data/home/apps/s~ldsdgidev/glen27.371429613087607751/django/forms/models.py", line 205, in new
opts.exclude, opts.widgets, formfield_callback)
File "/base/data/home/apps/s~ldsdgidev/glen27.371429613087607751/django/forms/models.py", line 145, in fields_for_model
opts = model._meta
AttributeError: type object 'UsersDB' has no attribute '_meta'
and I don't understand what I need to add to the UserDB class to get rid of the error.
Any help would be great!
This isn't anything to do with Python versions, or metaclasses.
ModelForms only work with Django models. db.Model is the App Engine model class, not the Django one. You can't use a modelform with that class.
You mention django-nonrel in your question tags. That project allows you to use the Django models - subclasses of models.Model with the App Engine datastore. You probably want to do that.

Getting error "ImportError: Could not find 'input_readers' on path 'map reduce'" trying to start mapReduce job

I'm getting this error... "ImportError: Could not find 'input_readers' on path 'map reduce'" when trying to Run my map reduce job via the http://localhost:8080/mapreduce launcher page.
It looks like my problem is similar to this post, AppEngine mapper API import error. Unfortunately, no definitive answers were given.
I've simplified it down to this tiny testmapreduce.py:
from google.appengine.ext import db
class TestEntity(db.Model):
value = db.StringProperty()
def mapperhandler(test):
print test.value
return
And my mapreduce.yaml:
mapreduce:
- name: Simplest MapReduce
mapper:
handler: testmapreduce.mapperhandler
input_reader: mapreduce.input_readers.DatastoreInputReader
params:
- name: entity_kind
default: testmapreduce.TestEntity
One possible clue is the presence of __init__.py has no effect (whether in the project root, the mapreduce directory, or both). I'm sure I'm making a beginner mistake, but over the last couple of days I have read every bit of documentation, and all the examples I can find. Thanks.
UPDATE:
I get the same error trying to invoke it via...
control.start_map(
"Give awards",
"testmapreduce.mapperhandler",
"mapreduce.input_readers.DatastoreInputReader",
{"entity_kind": "testmapreduce.TestEntity"},
shard_count=10)
UPDATE:
As requested, the stack trace -- let me know what else would be helpful...
ERROR 2011-10-16 17:09:27,216 _webapp25.py:464] Could not find 'input_readers' on path 'mapreduce'
Traceback (most recent call last):
File "/Develop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/_webapp25.py", line 701, in __call__
handler.get(*groups)
File "/Users/lc/PycharmProjects/mrtest/testmapreduce.py", line 22, in get
shard_count=10) # handle web form test interface
File "/Users/lc/PycharmProjects/mrtest/mapreduce/control.py", line 94, in start_map
transactional=transactional)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/handlers.py", line 811, in _start_map
mapper_input_reader_class = mapper_spec.input_reader_class()
File "/Users/lc/PycharmProjects/mrtest/mapreduce/model.py", line 393, in input_reader_class
return util.for_name(self.input_reader_spec)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/util.py", line 94, in for_name
module = for_name(module_name, recursive=True)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/util.py", line 102, in for_name
short_name, module_name))
ImportError: Could not find 'input_readers' on path 'mapreduce'
INFO 2011-10-16 22:09:27,253 dev_appserver.py:4247] "GET /giveawards HTTP/1.1" 500 -
This problem turned out to be that I was using the 2.7 version of the Python Interpreter in my local environment. When I switched to 2.5, it works fine.

Resources