URL Fetch AssertionError: 2 - google-app-engine

When using the asynchronous URL Fetch I get intermittent errors like this:
For 300 fetches I'm getting about 20 of theses. Am I doing something wrong? Or is this a bug?
Traceback (most recent call last):
File "/base/data/home/apps/s~culturemap-bulkmail/1.368407357562484358/bulkmail/views.py", line 83, in mailer
emailer.send(email, context)
File "/base/data/home/apps/s~culturemap-bulkmail/1.368407357562484358/bulkmail/mailers/amazon.py", line 91, in send
self.connection.send(email, msg.message().as_string())
File "/base/data/home/apps/s~culturemap-bulkmail/1.368407357562484358/bulkmail/mailers/amazon.py", line 49, in send
headers=self.headers,
File "/python27_runtime/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 339, in make_fetch_call
rpc.make_call('Fetch', request, response, _get_fetch_result, allow_truncated)
File "/python27_runtime/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 519, in make_call
assert self.__rpc.state == apiproxy_rpc.RPC.IDLE, repr(self.state)
AssertionError: 2
in init:
self.fetcher = urlfetch.create_rpc(deadline=60)
in method that gets called several times:
also line that errors out:
fetch = urlfetch.make_fetch_call(
self.fetcher,
'https://email.us-east-1.amazonaws.com/',
payload=form_data,
method=urlfetch.POST,
headers=self.headers,
)
I guess I should create a new rpc instead of reusing it?

Show some of your code, from the error it looks like you're reusing an existing active RPC object.

Related

prawcore.exceptions.NotFound: received 404 HTTP response

Hey guys I'm having trouble figuring this out:
Traceback (most recent call last):
File "SOLIS.py", line 62, in <module>
com_df1 = scraper.comments(sub_id_list,keywords1,keywords2)
File "/Users/CarranzaEE/Desktop/CDS 491:DEXIS/Reddit_SOLIS/Reddit_SOLIS.py", line 252, in comments
if((comment.author is None) or (hasattr(comment.author, 'id') != True)):
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/models/reddit/base.py", line 34, in __getattr__
self._fetch()
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/models/reddit/redditor.py", line 171, in _fetch
data = self._fetch_data()
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/models/reddit/redditor.py", line 168, in _fetch_data
return self._reddit.request("GET", path, params)
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/praw/reddit.py", line 765, in request
json=json,
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/prawcore/sessions.py", line 339, in request
url=url,
File "/Users/CarranzaEE/anaconda3/lib/python3.7/site-packages/prawcore/sessions.py", line 265, in _request_with_retries
raise self.STATUS_EXCEPTIONS[response.status_code](response)
prawcore.exceptions.NotFound: received 404 HTTP response
This error happens when I try to use author (an instance of Redditor) to try and call author.id. PRAW documents also say that Shadowbanned accounts are treated the same as non-existent accounts, meaning that they will not have any attributes, and that will only return name and is_suspended.
But is_suspended is no longer an attribute provided by reddit api. So I'm having trouble trying to handle cases where the account is a shawdowbanned account. Any tips or ideas?

Error while calling cloud framework endpoint api

I am getting below error while accessing the api via explorer '_ah/api/explorer' or when api is called from the app.
Below is relevant piece from app.yaml and api implementation.
From other questions on stackoverflow it looks like this could happen if endpoint config is not deployed using gcloud service-management but event after uploading gatewayv1openapi.json I am getting similar error.
Is there something i need to change in api code or app.yaml? One more thing is that I have uploaded exact same codebase (only difference is other project id instead of ixp-test in app.yaml) to another old project and it works fine. Any newly created project is giving this error.
/_ah/api/gateway/v1/invoke?alt=json
no scheduler thread, scheduler.run() will be invoked by report(...) (/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/lib/endpoints_management/control/client.py:235)
Traceback (most recent call last):
File "/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/lib/endpoints_management/control/client.py", line 231, in start
self._thread.start()
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/background_thread/background_thread.py", line 108, in start
start_new_background_thread(self.bootstrap, ())
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/api/background_thread/background_thread.py", line 87, in start_new_background_thread
raise ERROR_MAPerror.application_error
FrontendsNotSupported
.
.
.
(/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py:263)
Traceback (most recent call last):
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 96, in LoadObject
__import(cumulative_path)
File "/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/main.py", line 18, in
api = endpoints.api_server([GatewayApi])
File "/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/lib/endpoints/apiserving.py", line 514, in api_server
controller)
File "/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/lib/endpoints_management/control/wsgi.py", line 126, in add_all
a_service = loader.load()
File "/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/lib/endpoints_management/control/service.py", line 111, in load
return self._load_func(**kw)
File "/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/lib/endpoints_management/config/service_config.py", line 79, in fetch_service_config
_log_and_raise(Exception, message_template.format(status_code))
File "/base/data/home/apps/s~ixp-test/20171013t215957.404774686885375478/server/lib/endpoints_management/config/service_config.py", line 127, in _log_and_raise
raise exception_class(message)
Exception: Fetching service config failed (status code 403)
Below is the relevant config
app.yaml
runtime: python27
api_version: 1
threadsafe: false
automatic_scaling:
max_idle_instances: 0
env_variables:
ENDPOINTS_SERVICE_NAME: ixp-test.appspot.com
ENDPOINTS_SERVICE_VERSION: 2017-08-13r0
API
#endpoints.api(name='gateway', version='v1')
class GatewayApi(remote.Service):
#endpoints.method(
GatewayRequest,
GatewayResponse,
path='invoke',
http_method='POST',
name='invoke')
def invoke(self, request):
pass
#endpoints.method(
GatewayRequest,
GatewayResponse,
path='invokeSecure',
http_method='POST',
name='invokeSecure',
scopes=[endpoints.EMAIL_SCOPE])
def invoke_secure(self, request):
pass
This might well happen if the service name in the gatewayv1openapi.json doesn't match the service name in the app.yaml; can you double-check that?
Managed to resolve it. Issue was due to not updating
ENDPOINTS_SERVICE_VERSION: 2017-08-13r0
for new project. Value from old deployed project for which it was working caused the error.
from getting started tutorial
gcloud service-management deploy echov1openapi.json
Service Configuration [2017-02-13-r2] uploaded for service [example-project.appspot.com]

TemplateDoesNotExist error for a simple ModelAdmin example

I've used Wagtail's ModelAdmin on a few other projects with great success, but this is the first time I'm attempting to use it with a model inheriting from Page.
I copied the simple example template and removed the parts that didn't relate to my BlogPageModel.
The link to the ModelAdmin appears as expected, but when I click it, I get TemplateDoesNotExist at /admin/blog/blogpage/.
I don't remember having to set up a template before. Is there a setting I'm missing?
Wagtail: 1.9 /
Django: 1.10.5 /
Python: 3.5.2
Update: I tried it with a model that inherits from models.Model as well with the same result.
Traceback follows:
Internal Server Error: /admin/blog/blogpage/
Traceback (most recent call last):
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 39, in inner
response = get_response(request)
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/core/handlers/base.py", line 217, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/core/handlers/base.py", line 215, in _get_response
response = response.render()
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/template/response.py", line 109, in render
self.content = self.rendered_content
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/template/response.py", line 84, in rendered_content
template = self.resolve_template(self.template_name)
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/template/response.py", line 66, in resolve_template
return select_template(template, using=self.using)
File "/Users/username/.virtualenv/lib/python3.5/site-packages/django/template/loader.py", line 53, in select_template
raise TemplateDoesNotExist(', '.join(template_name_list), chain=chain)
django.template.exceptions.TemplateDoesNotExist: modeladmin/blog/blogpage/index.html, modeladmin/blog/index.html, modeladmin/index.html
[21/Feb/2017 07:56:38] "GET /admin/blog/blogpage/ HTTP/1.1" 500 110901
I suspect you've missed out adding 'wagtail.contrib.modeladmin' to INSTALLED_APPS.

pyVmomi exception on manager.CreateImportSpec

I am trying to deploy ovf using pyVmomi. I am trying to follow this example: https://github.com/vmware/pyvmomi-community-samples/blob/master/samples/deploy_ovf.py.
There are two VCenter setups and there are differences. On one VCenter, I can deploy the VM. On the other, I have a failure as below:
>>> spec_result = manager.CreateImportSpec(ovfd, rp_obj, datastore_obj,spec_params)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/VmomiSupport.py", line 570, in <lambda>
self.f(*(self.args + (obj,) + args), **kwargs)
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/VmomiSupport.py", line 376, in _InvokeMethod
return self._stub.InvokeMethod(self, info, args)
File "/Users/aitang/virenvs/py343/lib/python3.4/site-packages/pyVmomi/SoapAdapter.py", line 1350, in InvokeMethod
raise obj # pylint: disable-msg=E0702
pyVmomi.VmomiSupport.InvalidType: (vmodl.fault.InvalidType) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = '',
faultCause = <unset>,
faultMessage = (vmodl.LocalizableMessage) [],
argument = 'ManagedObjectReference'
}
Can someone point to the way on how I can troubleshoot this error? I tried to look at what this function does but couldn't find it.
I tried to set pdb trace, and it turns out that the reason it fails is that this comes back with a 500 response. On the VCenter that worked, it returns a 200 response. So why would the same call return differently?
I used the wrong datastore. The one I used was not of type "Datastore":
>>> type(datastore_obj)
<class 'pyVmomi.VmomiSupport.vim.StoragePod'>
When I used one of its childEntity, it worked:
>>> type(datastore_obj.childEntity[0])
<class 'pyVmomi.VmomiSupport.vim.Datastore'>
Here is what helped with debugging: I put a breakpoint in SoapAdapter.py. When 500 is returned to me, it entered pdb, and I obtained my own request, which contained this line at the bottom:
<datastore type="StoragePod">datastore-1</datastore>
When I compared this with the other VCenter that worked, I saw this difference:
<datastore type="Datastore">datastore-10</datastore>

Attribute error while exporting data from app engine using Remote APi

Use the example code from app engine will give an attribute error. The more strange thing is,
When the batch_size is 100, the first fetch will give an error while if it were set to 10, the second fetch will give the error, when the batch_size is 1, the 25th fetch will give the error. Is it due to the problem of remote API?
Python version: 2.7
App engine sdk version: 1.9.6
query = MyModel.all()
entities = query.fetch(100)
while entities:
for entity in entities:
# Do something with entity
query.with_cursor(query.cursor())
entities = query.fetch(100)
error message:
Traceback (most recent call last):
File "migrate.py", line 77, in <module>
entities = query.fetch(batch_size)
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 2157, in fetch
return list(self.run(limit=limit, offset=offset, **kwargs))
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 2326, in next
return self.__model_class.from_entity(self.__iterator.next())
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 1435, in from_entity
entity_values = cls._load_entity_values(entity)
File "/home/kamel/Library/google_appengine/google/appengine/ext/db/__init__.py", line 1413, in _load_entity_values
value = prop.make_value_from_datastore(value)
File "/home/kamel/labola/src/model/properties.py", line 295, in make_value_from_datastore
return pickle.loads(value)
File "/usr/lib/python2.7/pickle.py", line 1382, in loads
return Unpickler(file).load()
File "/usr/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1083, in load_newobj
obj = cls.__new__(cls, *args)
AttributeError: class Reference has no attribute '__new__
I encountered the same issue when trying to unpickle python3 pickles under python2. The problem was linked to new-style classes becoming default in python3. (source)
Solution for me was to replace class AClass: by class AClass(object):

Resources