Running locally with GAE Python second generation - google-app-engine

I've been using GAE/P 1st gen for many years and am making the jump to my first GAE/P 2nd gen project.
I'm trying to run it locally like I did with dev_appserver.py for 1st gen apps, but the docs are very confusing in this respect.
I can run Flask on its own (python main.py) but static files don't work (since Flask knows nothing about app.yaml) and the datastore won't work either.
The GAE docs say that dev_appserver.py should work for 2nd gen (except on Windows but I'm on Mac) but when I run it, I get the error below.
This happens when I run dev_appserver.py with Python 2 or 3. I've also set CLOUDSDK_PYTHON to my Python 2 binary as instructed.
So how the heck can I run my GAE/P 2nd gen project locally in way that static files and the datastore work?!?
$ python ~/google-cloud-sdk/bin/dev_appserver.py --application=myapp --support_datastore_emulator=False ./app.yaml
INFO 2019-02-12 00:39:21,885 devappserver2.py:278] Skipping SDK update check.
INFO 2019-02-12 00:39:23,128 api_server.py:275] Starting API server at: http://localhost:51187
INFO 2019-02-12 00:39:23,139 instance_factory.py:71] Detected Python 3.6.5
INFO 2019-02-12 00:39:24,459 instance_factory.py:205] Using pip to install dependency libraries; pip stdout is redirected to /var/folders/yd/xr84fg0946l72fp_cc6j9l540000gr/T/tmpj3SEUZ
INFO 2019-02-12 00:39:24,459 instance_factory.py:211] Running /var/folders/yd/xr84fg0946l72fp_cc6j9l540000gr/T/tmpLtysT3/bin/pip install --upgrade pip
INFO 2019-02-12 00:39:24,469 stub_util.py:357] Applying all pending transactions and saving the datastore
INFO 2019-02-12 00:39:24,469 stub_util.py:360] Saving search indexes
Traceback (most recent call last):
File "/.../google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 96, in <module>
_run_file(__file__, globals())
File "/.../google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 90, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 600, in <module>
main()
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 588, in main
dev_server.start(options)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 360, in start
options.api_host, apiserver.port, wsgi_request_info_)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py", line 248, in start
ssl_port)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py", line 384, in _create_module
ssl_port=ssl_port)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 1309, in __init__
super(AutoScalingModule, self).__init__(**kwargs)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 598, in __init__
self._module_configuration)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 231, in _create_instance_factory
module_configuration=module_configuration)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/instance_factory.py", line 98, in __init__
self._SetupVirtualenvFromConfiguration()
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/instance_factory.py", line 143, in _SetupVirtualenvFromConfiguration
self._venv_dir, requirements_file.name)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/instance_factory.py", line 228, in _SetupVirtualenv
cls._RunPipInstall(venv_dir, requirements_file_name)
File "/.../google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/instance_factory.py", line 212, in _RunPipInstall
pip_proc = subprocess.Popen(pip_cmd, stdout=pip_out)
File "/usr/local/Cellar/python#2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 394, in __init__
errread, errwrite)
File "/usr/local/Cellar/python#2/2.7.15/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1047, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

After a good night's sleep, I was able to figure out the problem...
I was running dev_appserver.py from within my virtual environment. For some reason, this prevented dev_appserver.py from being able to find pip (needed to install the packages in my requirements.txt).
Exiting my virtual environment before running dev_appserver.py fixed the problem.

Related

Resolve the error while using 'yarn db:start' command

I am trying to setup a ReactJS workspace in VSCode for Parabol. While executing the command yarn db:start, I am receiving the below exception.
yarn run v1.22.11
$ docker-compose -f docker/dev.yml up -d
Traceback (most recent call last):
File "docker\api\client.py", line 214, in _retrieve_server_version
File "docker\api\daemon.py", line 181, in version
File "docker\utils\decorators.py", line 46, in inner
File "docker\api\client.py", line 237, in _get
File "requests\sessions.py", line 543, in get
File "requests\sessions.py", line 530, in request
File "requests\sessions.py", line 643, in send
File "requests\adapters.py", line 439, in send
File "urllib3\connectionpool.py", line 670, in urlopen
File "urllib3\connectionpool.py", line 392, in _make_request
File "http\client.py", line 1255, in request
File "http\client.py", line 1301, in _send_request
File "http\client.py", line 1250, in endheaders
File "http\client.py", line 1010, in _send_output
File "http\client.py", line 950, in send
File "docker\transport\npipeconn.py", line 32, in connect
File "docker\transport\npipesocket.py", line 23, in wrapped
File "docker\transport\npipesocket.py", line 72, in connect
File "docker\transport\npipesocket.py", line 52, in connect
pywintypes.error: (2, 'CreateFile', 'The system cannot find the file specified.')
I am pretty new to ReactJS and have little to no idea on what to do to setup this workspace. Would be much grateful for any help.
Bruce here from Parabol. Thanks for your interest in our app!
For the issue you have seen, can you check:
You have Docker up and running
You run yarn db:start from the root of the repository
Feel free to create an issue in our repo and we will be more than happy to help you!

Error downloading App Engine source code with appcfg.py

I try to download my code from google using:
appcfg.py download_app -A "canada-math" ./trunk3
I have the old appengine installed under my home dir, which you can see in the error output below.
appcfg.py download_app -A "canada-math" ./trunk3
I get this:
stephen#stephen-AO725:~/webprog$ appcfg.py download_app -A "canada-math" ./trunk3
05:06 PM Host: appengine.google.com
05:06 PM Fetching file list...
Traceback (most recent call last):
File "/home/stephen/webprog/google_appengine/appcfg.py", line 171, in <module>
run_file(__file__, globals())
File "/home/stephen/webprog/google_appengine/appcfg.py", line 167, in run_file
execfile(script_path, globals_)
File "/home/stephen/webprog/google_appengine/google/appengine/tools/appcfg.py", line 4282, in <module>
main(sys.argv)
File "/home/stephen/webprog/google_appengine/google/appengine/tools/appcfg.py", line 4273, in main
result = AppCfgApp(argv).Run()
File "/home/stephen/webprog/google_appengine/google/appengine/tools/appcfg.py", line 2409, in Run
self.action(self)
File "/home/stephen/webprog/google_appengine/google/appengine/tools/appcfg.py", line 4003, in __call__
return method()
File "/home/stephen/webprog/google_appengine/google/appengine/tools/appcfg.py", line 2861, in DownloadApp
DoDownloadApp(rpcserver, out_dir, app_id, module, app_version)
File "/home/stephen/webprog/google_appengine/google/appengine/tools/appcfg.py", line 1373, in DoDownloadApp
result = rpcserver.Send('/api/files/list', **url_args)
File "/home/stephen/webprog/google_appengine/google/appengine/tools/appengine_rpc.py", line 393, in Send
f = self.opener.open(req)
File "/usr/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
TypeError: do_open() got an unexpected keyword argument 'context'
The simplest fix is likely to download the source code by running appcfg.py in Cloud Shell. That will ensure that your appcfg.py and Python versions are compatible with the App Engine API.
Depending on your specific versions of appcfg and Python, you may have a version mismatch. Also keep in mind that appcfg is deprecated. See also:
SDK is incompatible with python 2.7.9
Deploying Google app engine with python failed

OSX dev_appserver.py file not accessible: '/System/Library/CoreServices/SystemVersion.plist'

I did a gcloud components update 2 days ago and started getting this error when i run dev_appserver.py
(venv) myusername#mymachine:~/projects/myproject$ dev_appserver.py ./ --host 0.0.0.0 --port 8002 --enable_console --env_var GCS_TOKEN=ya29........YJDQAnp772B0
INFO 2019-03-13 23:45:31,205 devappserver2.py:278] Skipping SDK update check.
INFO 2019-03-13 23:45:31,268 api_server.py:275] Starting API server at: http://localhost:64587
INFO 2019-03-13 23:45:31,319 dispatcher.py:256] Starting module "default" running at: http://0.0.0.0:8002
INFO 2019-03-13 23:45:31,325 admin_server.py:150] Starting admin server at: http://localhost:8000
/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/mtime_file_watcher.py:182: UserWarning: There are too many files in your application for changes in all of them to be monitored. You may have to restart the development server to see some changes to your files.
'There are too many files in your application for '
INFO 2019-03-13 23:45:35,237 instance.py:294] Instance PID: 29760
appengine_config
requests.__version__ 2.21.0
Appengine config done
4
ERROR 2019-03-13 23:45:35,986 wsgi.py:263]
Traceback (most recent call last):
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/Users/myusername/projects/myproject/main.py", line 34, in <module>
from bp_includes.lib.error_handler import handle_error
File "/Users/myusername/projects/myproject/bp_includes/lib/error_handler.py", line 15, in <module>
from bp_includes.lib import jinja_bootstrap
File "/Users/myusername/projects/myproject/bp_includes/lib/jinja_bootstrap.py", line 7, in <module>
from src.handlers.utils import ordinal
File "/Users/myusername/projects/myproject/src/handlers/utils.py", line 14, in <module>
from lib.pytz.gae import pytz
File "/Users/myusername/projects/myproject/lib/pytz/__init__.py", line 29, in <module>
from pkg_resources import resource_stream
File "/Users/myusername/projects/myproject/lib/pkg_resources/__init__.py", line 1022, in <module>
class Environment(object):
File "/Users/myusername/projects/myproject/lib/pkg_resources/__init__.py", line 1025, in Environment
def __init__(self, search_path=None, platform=get_supported_platform(),
File "/Users/myusername/projects/myproject/lib/pkg_resources/__init__.py", line 263, in get_supported_platform
plat = get_build_platform()
File "/Users/myusername/projects/myproject/lib/pkg_resources/__init__.py", line 472, in get_build_platform
INFO 2019-03-13 23:45:36,002 module.py:861] default: "GET /_ah/warmup HTTP/1.1" 500 -
version = _macosx_vers()
File "/Users/myusername/projects/myproject/lib/pkg_resources/__init__.py", line 439, in _macosx_vers
version = platform.mac_ver()[0]
File "/Users/myusername/projects/myproject/venv/lib/python2.7/platform.py", line 764, in mac_ver
info = _mac_ver_xml()
File "/Users/myusername/projects/myproject/venv/lib/python2.7/platform.py", line 741, in _mac_ver_xml
pl = plistlib.readPlist(fn)
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plistlib.py", line 75, in readPlist
pathOrFile = open(pathOrFile)
File "/usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/stubs.py", line 284, in __init__
raise IOError(errno.EACCES, 'file not accessible', filename)
IOError: [Errno 13] file not accessible: '/System/Library/CoreServices/SystemVersion.plist'
I originally installed gcloud with brew cask install google-cloud-sdk in case that's relevant
It looks like the problem was introduced in gcloud 238.0.0 probably with gcloud app Python Extensions 1.9.84. You can downgrade gcloud to lower version with:
gcloud components update --version 237.0.0
I got it to work by doing the following in appengine_config.py, but I'm hoping that someone has a better solution
try:
from google.appengine.tools.devappserver2.python.runtime.stubs import FakeFile
FakeFile._allowed_dirs.update(['/System/Library/CoreServices/'])
except ImportError:
pass
you can delete the lib folder where you requirements are installed and reinstall it worked for me
pip install -t "lib" -r requirements.txt

gcloud deploy error: Image with tag google/docker-registry was not found

I'm trying to deploy this project https://github.com/GoogleCloudPlatform/appengine-nodejs-quickstart by using this command
gcloud --project poised-graph-758 preview app deploy --server preview.appengine.google.com .
but i get this error:
Updating module [default] from file [/Users/kannix/Desktop/tmp/appengine-nodejs-quickstart/app.yaml]
11:36 PM Host: preview.appengine.google.com
{bucket: vm-containers.poised-graph-758.appspot.com, path: /containers}
Traceback (most recent call last):
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/gcloud/gcloud.py", line 153, in <module>
main()
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/gcloud/gcloud.py", line 149, in main
_cli.Execute()
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/calliope/cli.py", line 381, in Execute
post_run_hooks=self.__post_run_hooks, kwargs=kwargs)
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/calliope/frontend.py", line 274, in _Execute
pre_run_hooks=pre_run_hooks, post_run_hooks=post_run_hooks)
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/calliope/backend.py", line 887, in Run
result = command_instance.Run(args)
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/calliope/exceptions.py", line 78, in TryFunc
return func(*args, **kwargs)
File "/Users/kannix/google-cloud-sdk/lib/googlecloudsdk/appengine/app_commands/deploy.py", line 154, in Run
self.__Run(args, project, deployables)
File "/Users/kannix/google-cloud-sdk/lib/googlecloudsdk/appengine/app_commands/deploy.py", line 207, in __Run
info.runtime, path, bucket)
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/appengine/lib/images/push.py", line 134, in BuildAndPushDockerImage
boto_path=config.BOTO_PATH)) as r:
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/appengine/lib/images/registry.py", line 148, in __enter__
self.Start()
File "/Users/kannix/google-cloud-sdk/./lib/googlecloudsdk/appengine/lib/images/registry.py", line 97, in Start
self._registry.Start()
File "/Users/kannix/google-cloud-sdk/platform/google_appengine/google/appengine/tools/docker/containers.py", line 480, in Start
self._image.Build()
File "/Users/kannix/google-cloud-sdk/platform/google_appengine/google/appengine/tools/docker/containers.py", line 309, in Build
raise ImageError('Image with tag %s was not found' % self.tag)
google.appengine.tools.docker.containers.ImageError: Image with tag google/docker-registry was not found
I tried to omit the --server preview.appengine.google.com parameter but it results in the same error
You need to pull the google/docker-registry image to resolve this error:
docker pull google/docker-registry
Note: if you receive the following error when trying to redeploy:
docker.docker.errors.APIError: 500 Server Error: Internal Server Error ("Invalid registry endpoint https://192.168.59.103:49153/v1/: Get https://192.168.59.103:49153/v1/_ping: EOF. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add `--insecure-registry 192.168.59.103:49153` to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/192.168.59.103:49153/ca.crt")
Ensure that your docker daemon has been setup with the 'insecure registry' option (you will need to re-pull the google/docker-registry after the following steps) . You can do this with the following, (which was an answer on https://github.com/docker/docker/issues/8887):
$ boot2docker delete #removes old image
$ rm -f ~/.ssh/id_boot2docker* # remove old keys
$ boot2docker init #generates new keys, cert
$ boot2docker up
$ boot2docker ssh
$ # add EXTRA_ARGS="--insecure-registry <YOUR INSECURE HOST>"
$ # to /var/lib/boot2docker/profile
$ sudo /etc/init.d/docker restart
Try running this first:
gcloud preview app setup-managed-vms
Also make sure you're running Docker 1.3.0, not 1.3.1.
Docker 1.4.0 also doesn't work at this moment.
Deploying with --server preview.appengine.google.com did it for me.

Simple Application in Go using Google App Engine SDK

When I follow Google-Developers' tutorial for creating a simple application in Go using App Engine SDK for Go, while running the command
goapp serve
I receive the following error:
Traceback (most recent call last):
File "/home/kamal/go_appengine/dev_appserver.py", line 82, in <module>
_run_file(__file__, globals())
File "/home/kamal/go_appengine/dev_appserver.py", line 78, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/home/kamal/go_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 1001, in <module>
main()
File "/home/kamal/go_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 994, in main
dev_server.start(options)
File "/home/kamal/go_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 754, in start
options.config_paths)
File "/home/kamal/go_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 632, in __init__
module_configuration = ModuleConfiguration(config_path)
File "/home/kamal/go_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 97, in __init__
self._config_path)
File "/home/kamal/go_appengine/google/appengine/tools/devappserver2/application_configuration.py", line 316, in _parse_configuration
config, files = appinfo_includes.ParseAndReturnIncludePaths(f)
File "/home/kamal/go_appengine/google/appengine/api/appinfo_includes.py", line 81, in ParseAndReturnIncludePaths
appyaml = appinfo.LoadSingleAppInfo(appinfo_file)
File "/home/kamal/go_appengine/google/appengine/api/appinfo.py", line 1873, in LoadSingleAppInfo
listener.Parse(app_info)
File "/home/kamal/go_appengine/google/appengine/api/yaml_listener.py", line 226, in Parse
self._HandleEvents(self._GenerateEventParameters(stream, loader_class))
File "/home/kamal/go_appengine/google/appengine/api/yaml_listener.py", line 177, in _HandleEvents
raise yaml_errors.EventError(e, event_object)
google.appengine.api.yaml_errors.EventError: Unknown url handler type.
URLMap
auth_fail_action=redirect
static_dir=None
secure=default
http_headers=None
url=/.*
static_files=None
expiration=None
upload=None
api_endpoint=None
script=None
application_readable=None
position=None
login=optional
mime_type=None
require_matching_file=None
in "myapp/app.yaml", line 8, column 1
error while running dev_appserver.py: exit status 1
I don't know how to debug it. Just a hint would be sufficient.
Also, can it be because I have no web server installed? Do I have to install Apache or Nginx for goapp serve or go handles the server stuff itself?
The problem was indentation of script in handlers in .yaml file.
I initially did it like this:
handlers:
- url: /.*
script: _go_app
While it should be:
handlers:
- url: /.*
script: _go_app
It works now!

Resources