tipfy nosetest ImportError: No module named fancy_urllib - google-app-engine

I am trying to run nosetest using tipfy and google app engine but I keep getting an import error:
From the google_appengine directory I execute the following command (directory contains dev_appserver.py):
nosetests /Users/me/Documents/python/project/ --with-gae --without-sandbox
but I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/nosetests", line 8, in <module>
load_entry_point('nose==0.11.4', 'console_scripts', 'nosetests')()
File "/Library/Python/2.6/site-packages/nose-0.11.4-py2.6.egg/nose/core.py", line 117, in __init__
**extra_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/unittest.py", line 816, in __init__
self.parseArgs(argv)
File "/Library/Python/2.6/site-packages/nose-0.11.4-py2.6.egg/nose/core.py", line 134, in parseArgs
self.config.configure(argv, doc=self.usage())
File "/Library/Python/2.6/site-packages/nose-0.11.4-py2.6.egg/nose/config.py", line 323, in configure
self.plugins.configure(options, self)
File "/Library/Python/2.6/site-packages/nose-0.11.4-py2.6.egg/nose/plugins/manager.py", line 270, in configure
cfg(options, config)
File "/Library/Python/2.6/site-packages/nose-0.11.4-py2.6.egg/nose/plugins/manager.py", line 93, in __call__
return self.call(*arg, **kw)
File "/Library/Python/2.6/site-packages/nose-0.11.4-py2.6.egg/nose/plugins/manager.py", line 161, in simple
result = meth(*arg, **kw)
File "build/bdist.macosx-10.6-universal/egg/nosegae.py", line 84, in configure
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver.py", line 51, in <module>
import fancy_urllib
ImportError: No module named fancy_urllib
I can load the tipfy hello_world project without any errors and I have other app engine projects on the same machine, all running fine.
Using a mac os x 10.6.6 and I have both nose and nosegae installed. I have also tried to execute the same command from within the /Users/me/Documents/python/project/ folder but I get the same result

I had the same problem and here is my quick fix:
Modify this file "/usr/local/bin/dev_appserver.py"
......
if version_tuple == (2, 4):
sys.stderr.write('Warning: Python 2.4 is not supported; this program may '
'break. Please use version 2.5 or greater.\n')
#Start Change
#DIR_PATH = os.path.abspath(os.path.dirname(os.path.realpath(__file__)))
DIR_PATH = "/usr/local/google_appengine"
#End Change
SCRIPT_DIR = os.path.join(DIR_PATH, 'google', 'appengine', 'tools')
......
Works for me so far.

Try to run it with option:
--gae-lib-root=/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine
and please show Python sys.path.

I also ran into this problem using Nose / NoseGAE. I had no luck with trying various --gae-lib-root values, but I did eventually have luck patching dev_appserver.py (located in /usr/local/google_appengine/google/appengine/tools/ in my MacOS install) as follows:
...
try:
import distutils.util
except ImportError:
pass
# ----- start of new code -----
import os, sys
DIR_PATH = '/usr/local/google_appengine'
EXTRA_PATHS = [
DIR_PATH,
os.path.join(DIR_PATH, 'lib', 'antlr3'),
os.path.join(DIR_PATH, 'lib', 'django_0_96'),
os.path.join(DIR_PATH, 'lib', 'fancy_urllib'),
os.path.join(DIR_PATH, 'lib', 'ipaddr'),
os.path.join(DIR_PATH, 'lib', 'webob'),
os.path.join(DIR_PATH, 'lib', 'yaml', 'lib'),
os.path.join(DIR_PATH, 'lib', 'simplejson'),
os.path.join(DIR_PATH, 'lib', 'graphy'),
]
sys.path = EXTRA_PATHS + sys.path
# ----- end of new code -----
import dummy_thread
...
This closely follows some code in appcfg.py (fix_sys_paths()) mentioned in GAE issue ticket #3597. I suspect the issue lies in how Nose sets up the execution path, although I can't prove so at the moment.

It turns out that it is because of a UI bug when you set up the "Python Path" in the Preferences of GoogleAppengineLuncher. It needs an Enter to confirm the setting:
sudo port install python2.7
Then set the "Python Path" to
/opt/local/bin/python2.7
Enter to confirm
See here

If you are running stand alone scripts then before using any appengine stuff you gotta link to dirs
import sys
sys.path.append('/usr/local/google_appengine/')
sys.path.append('/usr/local/google_appengine/lib')
sys.path.append('/usr/local/google_appengine/lib/yaml/lib/')
if 'google' in sys.modules:
del sys.modules['google']

Nowadays (2016) GAE Python recommends loading the project libraries in the file at the root level entitled "appengine_config.py".
Thus, if you are still facing some obnoxious issues, pls make sure to also add the os dir path to load the library folder "lib" (as can be seen in my code below):
"""`appengine_config` gets loaded when starting a new application instance."""
import os
from google.appengine.ext import vendor
# insert `lib` as a site directory so our `main` module can load
# third-party libraries, and override built-ins with newer
# versions.
vendor.add(os.path.join(os.path.dirname(__file__), 'lib'))
After I did the above, I was able to run my tests successfully from IntelliJ (PyCharm).
Furthermore, please notice my test settings setup on intelliJ (PyCharm)
Hope the above can help some Python GAE devs since I had some challenges in getting the NoseGAE simple setup to work. Take care!

Related

Starting Go server on App Engine localhost throws error

Can anyone tell me why this is happening:
$ dev_appserver.py nmg_server
INFO 2017-07-08 17:15:37,369 application_configuration.py:461] No version specified. Generated version id: 20170708t171537
WARNING 2017-07-08 17:15:37,369 application_configuration.py:166] The Managed VMs runtime is deprecated, please consider migrating your application to use the Flexible runtime. See https://cloud.google.com/appengine/docs/flexible/python/migrating for more details.
INFO 2017-07-08 17:15:37,472 devappserver2.py:116] Skipping SDK update check.
INFO 2017-07-08 17:15:37,513 api_server.py:312] Starting API server at: http://localhost:54096
INFO 2017-07-08 17:15:37,517 api_server.py:938] Applying all pending transactions and saving the datastore
INFO 2017-07-08 17:15:37,517 api_server.py:941] Saving search indexes
Traceback (most recent call last):
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 103, in <module>
_run_file(__file__, globals())
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 97, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 381, in <module>
main()
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 369, in main
dev_server.start(options)
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 196, in start
options.api_host, apiserver.port, wsgi_request_info_, options.grpc_apis)
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py", line 223, in start
_module.start()
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 1647, in start
self._add_instance()
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 1799, in _add_instance
expect_ready_request=True)
File "/Users/dgaedcke/gcloud_tools/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/go_runtime.py", line 189, in new_instance
self._go_application.maybe_build()):
TypeError: maybe_build() takes exactly 2 arguments (1 given)
I'm trying to run my server for testing on localhost and it keeps exiting with this error
This appears to be a bug in Cloud SDK which is affecting dev_appserver.py when using with App Engine Managed VMs. It does not seem to be affecting App Engine Standard or App Engine Flex environments.
Until Google releases a new Cloud SDK with the fix, you can modify the CLOUD_SDK_INSTALL_DIR//platform/google_appengine/google/appengine/tools/devappserver2/go_managedvm.py file locally as shown below (added both the patchable unified diff as well as before/after just for convenience).
Also consider moving to App Engine Flex since Managed VMs are deprecated and will not be supported after October 27, 2017.
Warning: The Managed VMs beta environment (applications deployed with
vm:true) is deprecated and will be decommissioned. This page is for
users who are already using the flexible environment with vm:true in
their app.yaml and want to upgrade to the latest release. If you are
updating your application from the standard environment, see the
Migrating Services from the Standard Environment to the Flexible
Environment instead.
Diff in patchable format
--- /Users/tuxdude/google-cloud-sdk-orig/platform/google_appengine/google/appengine/tools/devappserver2go_managedvm.py 2017-07-08 11:11:11.000000000 -0700
+++ /Users/tuxdude/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/go_managedvm.py 2017-07-08 11:11:11.000000000 -0700
## -152,15 +152,9 ##
logging.debug('Build succeeded:\n%s\n%s', stdout, stderr)
self._go_executable = exe_name
- def maybe_build(self, maybe_modified_since_last_build):
+ def maybe_build(self):
"""Builds an executable for the application if necessary.
- Args:
- maybe_modified_since_last_build: True if any files in the application root
- or the GOPATH have changed since the last call to maybe_build, False
- otherwise. This argument is used to decide whether a build is Required
- or not.
-
Returns:
True if compilation was successfully performed (will raise
an exception if compilation was attempted but failed).
## -173,9 +167,6 ##
self._work_dir = tempfile.mkdtemp('appengine-go-bin')
atexit.register(_rmtree, self._work_dir)
- if self._go_executable and not maybe_modified_since_last_build:
- return False
-
if self._go_executable:
logging.debug('Rebuilding Go application due to source modification')
else:
Before:
def maybe_build(self, maybe_modified_since_last_build):
"""Builds an executable for the application if necessary.
Args:
maybe_modified_since_last_build: True if any files in the application root
or the GOPATH have changed since the last call to maybe_build, False
otherwise. This argument is used to decide whether a build is Required
or not.
Returns:
True if compilation was successfully performed (will raise
an exception if compilation was attempted but failed).
False if compilation was not attempted.
Raises:
BuildError: if building the executable fails for any reason.
"""
if not self._work_dir:
self._work_dir = tempfile.mkdtemp('appengine-go-bin')
atexit.register(_rmtree, self._work_dir)
if self._go_executable and not maybe_modified_since_last_build:
return False
if self._go_executable:
logging.debug('Rebuilding Go application due to source modification')
else:
logging.debug('Building Go application')
self._build()
return True
After:
def maybe_build(self):
"""Builds an executable for the application if necessary.
Returns:
True if compilation was successfully performed (will raise
an exception if compilation was attempted but failed).
False if compilation was not attempted.
Raises:
BuildError: if building the executable fails for any reason.
"""
if not self._work_dir:
self._work_dir = tempfile.mkdtemp('appengine-go-bin')
atexit.register(_rmtree, self._work_dir)
if self._go_executable:
logging.debug('Rebuilding Go application due to source modification')
else:
logging.debug('Building Go application')
self._build()
return True

GAE Error When Trying To Run App via Dispatch File

Trying to run my app using the dispatch file like so:
$ dev_appserver.py dispatch.yaml app1/app.yaml app2/app.yaml
gives me the following error:
Traceback (most recent call last):
File "/Users/usera/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 101, in <module>
_run_file(__file__, globals())
File "/Users/usera/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 97, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/Users/usera/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 1041, in <module>
main()
File "/Users/usera/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 1037, in main
dev_server.stop()
File "/Users/usera/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 845, in stop
metrics.GetMetricsLogger().Stop()
File "/Users/usera/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/metrics.py", line 117, in Stop
total_run_time = int((Now() - self._start_time).total_seconds())
TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'
I'm using the latest gcloud installation from here. Each service/module (e.g. app1/app2 in the command above) is written in go.
Content of my dispatch file:
application: my-app
– url: “*/app2/*”
module: app2
– url: “*/app1/*”
module: app1
My python version is 2.7 of course. Any ideas?
I just ran into this problem too, and I believe the issue is dev_appserver requires a default module to route requests which don't match any dispatch rules.
I was able to define a default module by removing the service attribute from the app.yaml for my default module, and removing the dispatch rule for that module.
In your case, remove the service attribute from app1/app.yaml (if present), and remove the dispatch rule which points to app1 in dispatch.yaml.
Hope this helps!

Nose GAE : Cannot import dev_appserver, but app engine is still in PYTHONPATH

I am getting the following error when trying to run the nosetest from my GAE project:
nosetests --nologcapture --with-gae --without-sandbox --gae-lib-root=/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine
but I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/nosetests", line 8, in <module>
load_entry_point('nose==1.3.4', 'console_scripts', 'nosetests')()
File "/Library/Python/2.7/site-packages/nose/core.py", line 121, in __init__
**extra_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/Library/Python/2.7/site-packages/nose/core.py", line 145, in parseArgs
self.config.configure(argv, doc=self.usage())
File "/Library/Python/2.7/site-packages/nose/config.py", line 346, in configure
self.plugins.configure(options, self)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 284, in configure
cfg(options, config)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 99, in __call__
return self.call(*arg, **kw)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 167, in simple
result = meth(*arg, **kw)
File "/Library/Python/2.7/site-packages/nosegae.py", line 87, in configure
from google.appengine.tools import old_dev_appserver as dev_appserver
ImportError: cannot import name old_dev_appserver
The sys.path reads:
'/Users/dsinha/Downloads/eclipse/plugins/org.python.pydev_3.9.0.201411111611/pysrc', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/antlr3', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/fancy_urllib', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/ipaddr', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/yaml-3.10', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/rsa', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1_modules', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/simplejson', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/django-1.4', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/goo...
so the app engine libraries should be getting pulled in. Actually its first failing to pull in dev_appserver, and then trying and failing to pull old_dev_appserver
The dir inside the appengine/lib file is:
bash-3.2$ ls
__init__.py appcfg.py backends_xml_parser.py dispatch_xml_parser.py handler_generator.py php_cli.py value_mixin.pyc
__init__.pyc appcfg_java.py boolean_action.py docker handler_generator.pyc queue_xml_parser.py web_xml_parser.py
adaptive_thread_pool.py appengine_rpc.py boolean_action.pyc dos_xml_parser.py jarfile.py queue_xml_parser.pyc web_xml_parser.pyc
api_server.py appengine_rpc.pyc bulkload_client.py download_appstats.py java_quickstart.py remote_api_shell.py xml_parser_utils.py
app_engine_config_exception.py appengine_rpc_httplib2.py bulkloader.py endpointscfg.py java_quickstart.pyc requeue.py xml_parser_utils.pyc
app_engine_config_exception.pyc augment_mimetypes.py cron_xml_parser.py gen_protorpc.py java_utils.py sdk_update_checker.py yaml_translator.py
app_engine_web_xml_parser.py augment_mimetypes.pyc dev-channel-js.js handler.py java_utils.pyc sdk_update_checker.pyc yaml_translator.pyc
app_engine_web_xml_parser.pyc backends_conversion.py devappserver2 handler.pyc os_compat.py value_mixin.py
bash-3.2$ pwd
/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/google/appengine/tools
I also tried to find the modules available inside the google.appengine.tools package:
>>> import pkgutil
>>> [name for _, name, _ in pkgutil.iter_modules(['testpkg'])]
[]
This problem started occuring after I upgraded to App Engine 1.9.10 (to use the async search features). In a problem that I think is related, when I try to run the debug server from PyDev, it just silently terminated on any page request (localhost:8080).
Running dev_appserver . from the command line works fine though.
Nose-GAE broke with App Engine 1.9.17: https://github.com/Trii/NoseGAE/issues/6
Downgrading to 1.9.15 made the problem go away temporarily while waiting for the issue to be resolved by nose-gae

GAE SDK 1.9.5 and an InvalidCertificateException

Trying to import testbed from GAE SDK 1.95 with Python2.7.8 on osX Maverics 10.9.5
and I'm getting a InvalidCertificateException error.
from google.appengine.ext import testbed
File "/usr/local/google_appengine/google/appengine/ext/testbed/__init__.py", line 120, in <module>
from google.appengine.api import urlfetch_stub
File "/usr/local/google_appengine/google/appengine/api/urlfetch_stub.py", line 34, in <module>
_fancy_urllib_InvalidCertException = fancy_urllib.InvalidCertificateException
AttributeError: 'module' object has no attribute 'InvalidCertificateException'
I looked at the fancy_url module and the InvalidCertificateException class is there, so I don't understand why it's not importing.
Apparently others have had the same error, so I attempt to correct it by deleting: urlfetch_cacerts.txt AND cacerts.txt from:
GoogleAppEngineLauncher/Contents/Resources/GoogleAppEngineDefault.bundle/Content‌​s/Resources/google_appengine/lib/cacerts/
Apparently the GAE installer creates a nested directory, this was fixed by copying the contents in:
cd /usr/local/google_appengine/lib
cp fancy_urllib/fancy_urllib/__init__.py fancy_urllib/__init__.py
This is how the module is incorrectly structured, it looks like these 2 init.py files are duplicate:
/usr/local/google_appengine/lib/fancy_urllib/__init__.py # this file is empty
/usr/local/google_appengine/lib/fancy_urllib/fancy_urllib/__init__.py # this file contains the functions.
FIXED THE ERROR

SyntaxError when trying to create a new app using django nonrel

I'm playing around with django-nonrel and google app engine when I get the following error trying to create a new app.
Traceback (most recent call last):
File "./manage.py", line 2, in <module>
from django.core.management import execute_manager
File "/home/levi/Projects/DoneBox/django/__init__.py", line 14
if svn_rev != u'SVN-unknown':
^
SyntaxError: invalid syntax
The file that caused the exception is included below.
VERSION = (1, 3, 0, 'final', 0)
def get_version():
version = '%s.%s' % (VERSION[0], VERSION[1])
if VERSION[2]:
version = '%s.%s' % (version, VERSION[2])
if VERSION[3:] == ('alpha', 0):
version = '%s pre-alpha' % version
else:
if VERSION[3] != 'final':
version = '%s %s %s' % (version, VERSION[3], VERSION[4])
from django.utils.version import get_svn_revision
svn_rev = get_svn_revision()
if svn_rev != u'SVN-unknown':
version = "%s %s" % (version, svn_rev)
return version
I've looked at this file in emacs and I can't see the problem, and I've tried searching google with no luck. Can someone please point me in the right direction?
(For those interested in what I'm doing, please see http://www.allbuttonspressed.com/projects/djangoappengine.)
running this code in buffer with ipython shebang
! /usr/bin/env ipython
-- coding: utf-8 --
extending the end like this:
print version
return version
get_version()
get "1.3" at the ipython prompt
similar thing with python shell fails like this
from django.utils.version import get_svn_revision
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named utils.version
assume some install/path issue
Just an FYI, the latest version of django-appengine is found here:
https://github.com/django-nonrel
However, the allbuttonspressed version should be working. You may not have your python environment set properly.

Resources