My directory structure in app engine is like this :
project
app.yaml
main.py
lib
markdown
markdown_files
Markdown is being imported in main.py by from lib.markdown import markdown . At this point dev server is starting to give me the following stack trace. I am wondering what's going wrong here !? I do have an init.py in lib.
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 2769, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 2654, in _Dispatch
base_env_dict=env_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 683, in Dispatch
base_env_dict=base_env_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1718, in Dispatch
self._module_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1622, in ExecuteCGI
reset_modules = exec_script(config, handler_path, cgi_path, hook)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 1310, in ExecuteOrImportScript
exec module_code in script_module.__dict__
File "/Users/nomadali/jeeqs/git/src/jeeqs.py", line 26, in <module>
from lib.markdown import markdown
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 676, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1845, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 676, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1717, in FindAndLoadModule
description)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 676, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1660, in LoadModuleRestricted
description)
File "/Users/nomadali/jeeqs/git/src/lib/markdown/__init__.py", line 161, in <module>
import preprocessors
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 676, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1845, in load_module
return self.FindAndLoadModule(submodule, fullname, search_path)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 676, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1717, in FindAndLoadModule
description)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 676, in Decorate
return func(self, *args, **kwargs)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py", line 1660, in LoadModuleRestricted
description)
File "/Users/nomadali/jeeqs/git/src/markdown/preprocessors.py", line 11, in <module>
ImportError: No module named markdown
INFO 2012-02-20 19:53:02,506 dev_appserver.py:2884] "GET /favicon.ico HTTP/1.0" 500 -
Update1
I believe this question is a dupe of Import Python module with PyImport_ImportModuleEx for a gedit plugin
The answer:
If you want to use markdown without modifying it then you're going to have to put it somewhere where the Python library expects it, such as in site-packages/. Otherwise, you will have to modify it to use relative imports instead of absolute imports.
Update 2
I resolved this issue by modifying my sys.path in app engine:
sys.path.append(os.path.join(os.path.dirname(file), 'lib'))
Thanks to Murat for help,
I believe this question is a dupe of Import Python module with PyImport_ImportModuleEx for a gedit plugin
The answer:
If you want to use markdown without modifying it then you're going to have to put it somewhere where the Python library expects it, such as in site-packages/. Otherwise, you will have to modify it to use relative imports instead of absolute imports.
Update 2 I resolved this issue by modifying my sys.path in app engine:
sys.path.append(os.path.join(os.path.dirname(file), 'lib'))
Thanks to Murat for help,
The file needs to be called __init__.py for that to work.
Adding the lib directory to the pythonpath also works.
If you want to use markdown without modifying it then you're going to have to put it somewhere where the Python library expects it, such as in site-packages/
This is incorrect, app engine has its own import mechanism and only packages that are explicitly white listed are allowed to be imported via this method.
What you want is for lib to be in your PYTHONPATH, the paths where Python looks stuff that you try to import. As you've discovered, one way to do that is to add lib to sys.path.
Creating an init.py file is not what you want, because that makes lib a package containing markdown. At that point, you should be able to do
from lib import markdown
although that's not the recommended way to do it.
Related
I'm tyring to use remote_api_shell.py to access the datastore of my Google App Engine application in the following way:
remote_api_shell.py -s url/of/server --secure
But I get the following error:
Traceback (most recent call last):
File "./remote_api_shell.py", line 133, in <module>
run_file(__file__, globals())
File "./remote_api_shell.py", line 129, in run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 160, in <module>
main(sys.argv)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 156, in main
oauth2=True)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/remote_api_shell.py", line 74, in remote_api_shell
secure=secure)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 768, in ConfigureRemoteApiForOAuth
rpc_server_factory=rpc_server_factory)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 835, in ConfigureRemoteApi
app_id = GetRemoteAppIdFromServer(server, path, rtok)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/ext/remote_api/remote_api_stub.py", line 569, in GetRemoteAppIdFromServer
response = server.Send(path, payload=None, **urlargs)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appengine_rpc_httplib2.py", line 245, in Send
url, method=method, body=payload, headers=headers)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/oauth2client/oauth2client/client.py", line 562, in new_request
redirections, connection_type)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1584, in request
(response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1332, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1268, in _conn_request
conn.connect()
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 1014, in connect
self.disable_ssl_certificate_validation, self.ca_certs)
File "/home/vagrant/google-cloud-sdk/platform/google_appengine/lib/httplib2/httplib2/__init__.py", line 80, in _ssl_wrap_socket
cert_reqs=cert_reqs, ca_certs=ca_certs)
File "/usr/lib/python2.7/ssl.py", line 487, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 243, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 405, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 8] _ssl.c:510: EOF occurred in violation of protocol
Any help would be appreciated.
The problem went away when I switched from using my custom domain name to my-app-id.appspot.com.
Some other considerations:
If you are using https, then your-app-id.appspot.com will not work out of the box. You have to enable it on the Google Apps page corresponding to the domain name. Yes, that means that you will actually have to have Google Apps setup for that domain.
If you have a multi-module setup, make sure you add remote_api: on to the builtins section of the app.yaml of the default module. Or if you have remote-api: on for another module, then use the url that skips the dispatch.yaml file and targets the module directly.
Below is a fragment of my code, i tried passing an instance variable from the class DiscussPage to anothe class quesAnsweredPage
class DiscussPage(webapp2.RequestHandler):
def post(self):
self.Ques_tit = "Content"
classname = questAnsweredPage()
classname.setValue(self.Ques_tit,self.Ques_tit)
class questAnsweredPage(webapp2.RequestHandler):
Can_I = "yes"
def setValue(self,que,title):
self.response.write(que)
The Code above returns the error:
Traceback (most recent call last):
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 1102, in __call__
return handler.dispatch()
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "C:\Program Files (x86)\Google\google_appengine\lib\webapp2-2.5.2\webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "B:\Eclipse World\Google App Engine\App Engine Dev\Jambite_Extended\JambitesCurrent\Jambitesupdated by bel\main.py", line 2686, in post
classname.setValue(self.Ques_tit,self.Ques_tit)
File "B:\Eclipse World\Google App Engine\App Engine Dev\Jambite_Extended\JambitesCurrent\Jambitesupdated by bel\main.py", line 2774, in setValue
self.response.write(que)
AttributeError: 'NoneType' object has no attribute 'write'
How do i resolve this error
You need to look at the signature of classes based on webapp2.RequestHandler ( specifically the init method)
From the docs http://webapp-improved.appspot.com/api/webapp2.html#webapp2.RequestHandler we get
class webapp2.RequestHandler(request=None, response=None)[source]
So when you instantiate an instance of a class inheriting from RequestHandler you need to initialise it correctly.
You code should read
classname = questAnsweredPage(self.request,self.response)
Now why you would use a variable name classname to hold an instance of a class is a whole separate style issue ;-)
The error is probably because self.response can only be called inside a get() or post() method.
It seems gae assigns very high IDs to the models.
When I download my entities, I get for some entries very big numbers. These were autogenerated in first place. Downloading them as csv is no problem. But deleting the existing data and re-uploading the same data throws an exception.
Exceeded maximum allocated IDs
Trace:
Traceback (most recent call last):
File "/opt/eclipse/plugins/org.python.pydev_2.7.5.2013052819/pysrc/pydevd.py", line 1397, in <module>
debugger.run(setup['file'], None, None)
File "/opt/eclipse/plugins/org.python.pydev_2.7.5.2013052819/pysrc/pydevd.py", line 1090, in run
pydev_imports.execfile(file, globals, locals) #execute the script
File "/home/kave/workspace/google_appengine/appcfg.py", line 171, in <module>
run_file(__file__, globals())
File "/home/kave/workspace/google_appengine/appcfg.py", line 167, in run_file
execfile(script_path, globals_)
File "/home/kave/workspace/google_appengine/google/appengine/tools/appcfg.py", line 4247, in <module>
main(sys.argv)
File "/home/kave/workspace/google_appengine/google/appengine/tools/appcfg.py", line 4238, in main
result = AppCfgApp(argv).Run()
File "/home/kave/workspace/google_appengine/google/appengine/tools/appcfg.py", line 2396, in Run
self.action(self)
File "/home/kave/workspace/google_appengine/google/appengine/tools/appcfg.py", line 3973, in __call__
return method()
File "/home/kave/workspace/google_appengine/google/appengine/tools/appcfg.py", line 3785, in PerformUpload
run_fn(args)
File "/home/kave/workspace/google_appengine/google/appengine/tools/appcfg.py", line 3676, in RunBulkloader
sys.exit(bulkloader.Run(arg_dict))
File "/home/kave/workspace/google_appengine/google/appengine/tools/bulkloader.py", line 4379, in Run
return _PerformBulkload(arg_dict)
File "/home/kave/workspace/google_appengine/google/appengine/tools/bulkloader.py", line 4244, in _PerformBulkload
loader.finalize()
File "/home/kave/workspace/google_appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 384, in finalize
self.increment_id(high_id_key)
File "/home/kave/workspace/google_appengine/google/appengine/tools/bulkloader.py", line 1206, in IncrementId
unused_start, end = datastore.AllocateIds(high_id_key, max=high_id_key.id())
File "/home/kave/workspace/google_appengine/google/appengine/api/datastore.py", line 1965, in AllocateIds
return AllocateIdsAsync(model_key, size, **kwargs).get_result()
File "/home/kave/workspace/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 612, in get_result
return self.__get_result_hook(self)
File "/home/kave/workspace/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1863, in __allocate_ids_hook
self.check_rpc_success(rpc)
File "/home/kave/workspace/google_appengine/google/appengine/datastore/datastore_rpc.py", line 1236, in check_rpc_success
raise _ToDatastoreError(err)
google.appengine.api.datastore_errors.BadRequestError: Exceeded maximum allocated IDs
Usually my Id's are around 26002 but the new id's since a few days ago are as big as 4948283361329150. These are causing problems now. (If I change them to lower values, its all fine, but i didn't generate these ids in first place) Why does GAE have such problems with its own generated ids?
Many Thanks
This is a known issue, fixed in the 1.8.2 or later SDKs.
Note, if you use bulkloader against the dev appserver those SDKs (1.8.2, 1.8.3) unfortunately have a separate bulkloader issue with that use case (see appcfg-py-upload-data-fails-in-google-app-engine-sdk-1-8-2) but not in production.
A popup on my desktop recently offered to update GoogleAppEngineLauncher, and I agreed to it.
My Python apps in PyCharm then showed "unrecognized reference" for "google" in from google.appengine.api import users or any other reference to "google.appengine.api".
So I launched GoogleAppEngineLauncher and clicked something, and it looked like it did an unzip or something. That made the PyCharm reference errors go away.
When I then tried to launch my app with my old configuration, it had a problem with "Additional options". Apparently it no longer recognizes "-p 8081" or "-c". So I changed them to "--port 8081" and "--clear_datastore". Now it terminates with the following stacktrace:
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 194, in <module>
_run_file(__file__, globals())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 190, in _run_file
execfile(script_path, globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 545, in <module>
main()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 535, in main
options = PARSER.parse_args()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1688, in parse_args
args, argv = self.parse_known_args(args, namespace)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1720, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1926, in _parse_known_args
start_index = consume_optional(start_index)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1866, in consume_optional
take_action(action, args, option_string)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1794, in take_action
action(self, namespace, argument_values, option_string)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/boolean_action.py", line 67, in __call__
raise ValueError('must be "yes" or "no", not %r' % values)
ValueError: must be "yes" or "no", not '.'
I tried running "helloworld" in GoogleAppEngineLauncher and got the following in the LogConsole:
*** Running dev_appserver with the following flags:
--skip_sdk_update_check=yes --port=8081 --admin_port=8000 --clear_datastore
Python command: /usr/bin/python2.7
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 194, in <module>
_run_file(__file__, globals())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/dev_appserver.py", line 190, in _run_file
execfile(script_path, globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 545, in <module>
main()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 535, in main
options = PARSER.parse_args()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1678, in parse_args
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1710, in parse_known_args
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1916, in _parse_known_args
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1856, in consume_optional
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1784, in take_action
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/boolean_action.py", line 67, in __call__
raise ValueError('must be "yes" or "no", not %r' % values)
ValueError: must be "yes" or "no", not '/Users/lindsay/Projects/PyCharm/Zephyr/gae-tutorial/helloworld'
This seems to be more or less the same error as from running my app in PyCharm.
I am under tremendous time pressure on this project, and this is completely blocking me. Any help will be most appreciated.
The solution to this problem was that configuration option -c has to be changed to --clear_datastore=yes.
Now I'm getting to another error, "InvalidCertificateException". I'll investigate that separately, and post a separate question if I can't solve it.
I tried to start web2py 1.98.1 using dev_appserver.py, but it cannot
find the "pickle" module.
Traceback (most recent call last): File
"/home/matthew/dev/sdks/google_appengine_1.5.2/google/appengine/
tools/dev_appserver.py", line 4144, in _HandleRequest
self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
File "/home/matthew/dev/sdks/google_appengine_1.5.2/google/appengine/
tools/dev_appserver.py", line 4049, in _Dispatch
base_env_dict=env_dict) File
"/home/matthew/dev/sdks/google_appengine_1.5.2/google/appengine/
tools/dev_appserver.py", line 616, in Dispatch
base_env_dict=base_env_dict) File
"/home/matthew/dev/sdks/google_appengine_1.5.2/google/appengine/
tools/dev_appserver.py", line 3120, in Dispatch
self._module_dict) File
"/home/matthew/dev/sdks/google_appengine_1.5.2/google/appengine/
tools/dev_appserver.py", line 3024, in ExecuteCGI
reset_modules = exec_script(handler_path, cgi_path, hook)
File "/home/matthew/dev/sdks/google_appengine_1.5.2/google/appengine/
tools/dev_appserver.py", line 2887, in ExecuteOrImportScript
exec module_code in script_module.dict File
"/home/matthew/dev/projects/poprop_cameron/project/src/
gaehandler.py", line 33, in
import pickle ImportError: No module named pickle
When I import it using the interpreter, it imports without error. Is
there a path I have to include somewhere?