pyVmomi exception on manager.CreateImportSpec - pyvmomi

I am trying to deploy ovf using pyVmomi. I am trying to follow this example: https://github.com/vmware/pyvmomi-community-samples/blob/master/samples/deploy_ovf.py.
There are two VCenter setups and there are differences. On one VCenter, I can deploy the VM. On the other, I have a failure as below:
>>> spec_result = manager.CreateImportSpec(ovfd, rp_obj, datastore_obj,spec_params)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/VmomiSupport.py", line 570, in <lambda>
self.f(*(self.args + (obj,) + args), **kwargs)
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/VmomiSupport.py", line 376, in _InvokeMethod
return self._stub.InvokeMethod(self, info, args)
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/SoapAdapter.py", line 1350, in InvokeMethod
raise obj # pylint: disable-msg=E0702
pyVmomi.VmomiSupport.InvalidType: (vmodl.fault.InvalidType) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = '',
faultCause = <unset>,
faultMessage = (vmodl.LocalizableMessage) [],
argument = 'ManagedObjectReference'
}
Can someone point to the way on how I can troubleshoot this error? I tried to look at what this function does but couldn't find it.
I tried to set pdb trace, and it turns out that the reason it fails is that this comes back with a 500 response. On the VCenter that worked, it returns a 200 response. So why would the same call return differently?

I used the wrong datastore. The one I used was not of type "Datastore":
>>> type(datastore_obj)
<class 'pyVmomi.VmomiSupport.vim.StoragePod'>
When I used one of its childEntity, it worked:
>>> type(datastore_obj.childEntity[0])
<class 'pyVmomi.VmomiSupport.vim.Datastore'>
Here is what helped with debugging: I put a breakpoint in SoapAdapter.py. When 500 is returned to me, it entered pdb, and I obtained my own request, which contained this line at the bottom:
<datastore type="StoragePod">datastore-1</datastore>
When I compared this with the other VCenter that worked, I saw this difference:
<datastore type="Datastore">datastore-10</datastore>

Related

prawcore.exceptions.NotFound: received 404 HTTP response

Hey guys I'm having trouble figuring this out:
Traceback (most recent call last):
File "SOLIS.py", line 62, in <module>
com_df1 = scraper.comments(sub_id_list,keywords1,keywords2)
File "/Users/CarranzaEE/Desktop/CDS 491:DEXIS/Reddit_SOLIS/Reddit_SOLIS.py", line 252, in comments
if((comment.author is None) or (hasattr(comment.author, 'id') != True)):
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/models/reddit/base.py", line 34, in __getattr__
self._fetch()
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/models/reddit/redditor.py", line 171, in _fetch
data = self._fetch_data()
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/models/reddit/redditor.py", line 168, in _fetch_data
return self._reddit.request("GET", path, params)
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/reddit.py", line 765, in request
json=json,
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/prawcore/sessions.py", line 339, in request
url=url,
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/prawcore/sessions.py", line 265, in _request_with_retries
raise self.STATUS_EXCEPTIONS[response.status_code](response)
prawcore.exceptions.NotFound: received 404 HTTP response
This error happens when I try to use author (an instance of Redditor) to try and call author.id. PRAW documents also say that Shadowbanned accounts are treated the same as non-existent accounts, meaning that they will not have any attributes, and that will only return name and is_suspended.
But is_suspended is no longer an attribute provided by reddit api. So I'm having trouble trying to handle cases where the account is a shawdowbanned account. Any tips or ideas?

No longer able to create an entity using the dev_appserver admin console

I cleared all entities from the datastore and cleared the memcache, created an entity using the API Explorer and then attempted to create another one of the same kind in the Datastore console running on localhost:
Internal Server Error
The server has either erred or is incapable of performing the requested operation.
Traceback (most recent call last):
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1536, in __call__
rv = self.handle_exception(request, response, e)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1530, in __call__
rv = self.router.dispatch(request, response)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/admin_request_handler.py", line 97, in dispatch
super(AdminRequestHandler, self).dispatch()
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py", line 760, in get
count=20)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py", line 121, in _get_entities
query = datastore.Query(kind, _namespace=namespace)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/api/datastore.py", line 1361, in __init__
datastore_errors.BadArgumentError)
File "/home/tom/work/google-cloud-sdk/platform/google_appengine/google/appengine/api/datastore_types.py", line 181, in ValidateString
raise exception('%s must not be empty.' % name)
BadArgumentError: kind must not be empty.
Here's the gcloud version info:
$ gcloud --version
Google Cloud SDK 157.0.0
app-engine-python 1.9.54
beta 2017.03.24
bq 2.0.24
core 2017.05.30
gcloud
gsutil 4.26
Any hints as to what might be wrong?
I think it's a bug in the Datastore console.
I was able to replicate the same error on my machine. If you look at the traceback in the exception, you see (fourth from the bottom):
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/admin/datastore_viewer.py", line 760, in get
count=20)
If you look at that file from line 735, you see:
def get(self, entity_key_string=None):
super(DatastoreEditRequestHandler, self).get(entity_key_string)
if entity_key_string:
entity_key = datastore.Key(entity_key_string)
entity_key_name = entity_key.name()
entity_key_id = entity_key.id()
namespace = entity_key.namespace()
kind = entity_key.kind()
entities = [datastore.Get(entity_key)]
parent_key = entity_key.parent()
if parent_key:
parent_key_string = _format_datastore_key(parent_key)
else:
parent_key_string = None
else:
entity_key = None
entity_key_string = None
entity_key_name = None
entity_key_id = None
namespace = self.request.get('namespace')
kind = self.request.get('kind')
entities, _ = _get_entities(kind,
namespace,
order=None,
start=0,
count=20) ### line 760 #######
...
In that last line, kind is supposed to be string containing the model name of the model you are working with. Instead, if you follow the traceback down into that function call, you see that kind is an empty string. I don't think there's much you can do to fix it at this time. I'd recommend using the interactive console for now to create/edit your Datastore entities, and just use the viewer to view them. I'm sure Google will fix this with an update soon.
Edit: I actually found a hacky fix if you really want the Datastore console to work, but I would still recommend just using the interactive console. The hacky fix is you manually enter the kind variable into the URL sent by the Datastore console.
When you click "Create New Entity" and get the error you show in your question, look at the url:
http://localhost:8000/datastore/edit?kind=&next=http%3A%2F%2Flocalhost%3A8000%2Fdatastore%3Fkind%3DUnique
You see ...:8000/datastore/edit?kind=&.... Right there, just type in the model name. For example, let's say your model name is User:
http://localhost:8000/datastore/edit?kind=User&next=http%3A%2F%2Flocalhost%3A8000%2Fdatastore%3Fkind%3DUnique
This brought me to the "Create new entity" form for class User (one of my models), and I was able to create a user with that form.
P.S. You may want to also change the model name on the very end of that url (where mine says Unique) to match the kind you are creating, because that model name is the kind that will be displayed after you create your new entity. So mine created a User, but then displayed the Unique entities (Unique is one of my model names).

Attribute error while exporting data from app engine using Remote APi

Use the example code from app engine will give an attribute error. The more strange thing is,
When the batch_size is 100, the first fetch will give an error while if it were set to 10, the second fetch will give the error, when the batch_size is 1, the 25th fetch will give the error. Is it due to the problem of remote API?
Python version: 2.7
App engine sdk version: 1.9.6
query = MyModel.all()
entities = query.fetch(100)
while entities:
for entity in entities:
# Do something with entity
query.with_cursor(query.cursor())
entities = query.fetch(100)
error message:
Traceback (most recent call last):
File "migrate.py", line 77, in <module>
entities = query.fetch(batch_size)
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 2157, in fetch
return list(self.run(limit=limit, offset=offset, **kwargs))
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 2326, in next
return self.__model_class.from_entity(self.__iterator.next())
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 1435, in from_entity
entity_values = cls._load_entity_values(entity)
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 1413, in _load_entity_values
value = prop.make_value_from_datastore(value)
File "/home/kamel/labola/src/model/properties.py", line 295, in make_value_from_datastore
return pickle.loads(value)
File "/usr/lib/python2.7/pickle.py", line 1382, in loads
return Unpickler(file).load()
File "/usr/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1083, in load_newobj
obj = cls.__new__(cls, *args)
AttributeError: class Reference has no attribute '__new__
I encountered the same issue when trying to unpickle python3 pickles under python2. The problem was linked to new-style classes becoming default in python3. (source)
Solution for me was to replace class AClass: by class AClass(object):

URL Fetch AssertionError: 2

When using the asynchronous URL Fetch I get intermittent errors like this:
For 300 fetches I'm getting about 20 of theses. Am I doing something wrong? Or is this a bug?
Traceback (most recent call last):
File "/base/data/home/apps/s~culturemap-bulkmail/1.368407357562484358/bulkmail/views.py", line 83, in mailer
emailer.send(email, context)
File "/base/data/home/apps/s~culturemap-bulkmail/1.368407357562484358/bulkmail/mailers/amazon.py", line 91, in send
self.connection.send(email, msg.message().as_string())
File "/base/data/home/apps/s~culturemap-bulkmail/1.368407357562484358/bulkmail/mailers/amazon.py", line 49, in send
headers=self.headers,
File "/python27_runtime/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 339, in make_fetch_call
rpc.make_call('Fetch', request, response, _get_fetch_result, allow_truncated)
File "/python27_runtime/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 519, in make_call
assert self.__rpc.state == apiproxy_rpc.RPC.IDLE, repr(self.state)
AssertionError: 2
in init:
self.fetcher = urlfetch.create_rpc(deadline=60)
in method that gets called several times:
also line that errors out:
fetch = urlfetch.make_fetch_call(
self.fetcher,
'https://email.us-east-1.amazonaws.com/',
payload=form_data,
method=urlfetch.POST,
headers=self.headers,
)
I guess I should create a new rpc instead of reusing it?
Show some of your code, from the error it looks like you're reusing an existing active RPC object.

Getting error "ImportError: Could not find 'input_readers' on path 'map reduce'" trying to start mapReduce job

I'm getting this error... "ImportError: Could not find 'input_readers' on path 'map reduce'" when trying to Run my map reduce job via the http://localhost:8080/mapreduce launcher page.
It looks like my problem is similar to this post, AppEngine mapper API import error. Unfortunately, no definitive answers were given.
I've simplified it down to this tiny testmapreduce.py:
from google.appengine.ext import db
class TestEntity(db.Model):
value = db.StringProperty()
def mapperhandler(test):
print test.value
return
And my mapreduce.yaml:
mapreduce:
- name: Simplest MapReduce
mapper:
handler: testmapreduce.mapperhandler
input_reader: mapreduce.input_readers.DatastoreInputReader
params:
- name: entity_kind
default: testmapreduce.TestEntity
One possible clue is the presence of __init__.py has no effect (whether in the project root, the mapreduce directory, or both). I'm sure I'm making a beginner mistake, but over the last couple of days I have read every bit of documentation, and all the examples I can find. Thanks.
UPDATE:
I get the same error trying to invoke it via...
control.start_map(
"Give awards",
"testmapreduce.mapperhandler",
"mapreduce.input_readers.DatastoreInputReader",
{"entity_kind": "testmapreduce.TestEntity"},
shard_count=10)
UPDATE:
As requested, the stack trace -- let me know what else would be helpful...
ERROR 2011-10-16 17:09:27,216 _webapp25.py:464] Could not find 'input_readers' on path 'mapreduce'
Traceback (most recent call last):
File "/Develop/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/_webapp25.py", line 701, in __call__
handler.get(*groups)
File "/Users/lc/PycharmProjects/mrtest/testmapreduce.py", line 22, in get
shard_count=10) # handle web form test interface
File "/Users/lc/PycharmProjects/mrtest/mapreduce/control.py", line 94, in start_map
transactional=transactional)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/handlers.py", line 811, in _start_map
mapper_input_reader_class = mapper_spec.input_reader_class()
File "/Users/lc/PycharmProjects/mrtest/mapreduce/model.py", line 393, in input_reader_class
return util.for_name(self.input_reader_spec)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/util.py", line 94, in for_name
module = for_name(module_name, recursive=True)
File "/Users/lc/PycharmProjects/mrtest/mapreduce/util.py", line 102, in for_name
short_name, module_name))
ImportError: Could not find 'input_readers' on path 'mapreduce'
INFO 2011-10-16 22:09:27,253 dev_appserver.py:4247] "GET /giveawards HTTP/1.1" 500 -
This problem turned out to be that I was using the 2.7 version of the Python Interpreter in my local environment. When I switched to 2.5, it works fine.

Resources