I'm trying to test the new managed VM feature, following this tutorial (using OS X)
The boot2docker installation it's okay.
Executing $(boot2docker shellinit) give the following output:
Writing /Users/guilhermetorres/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/guilhermetorres/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/guilhermetorres/.boot2docker/certs/boot2docker-vm/key.pem
export DOCKER_HOST=tcp://192.168.59.103:2376
export DOCKER_CERT_PATH=/Users/guilhermetorres/.boot2docker/certs/boot2docker-vm
export DOCKER_TLS_VERIFY=1
boot2docker status print running
and boot2docker version print Boot2Docker-cli version: v1.5.0
Although when I try to run my application using gcloud --verbosity debug preview app run
I got this error:
ERROR 2015-03-05 00:36:39,816 containers.py:742] Failed to connect to Docker Daemon due to: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
You can see the EV variables:
DEBUG 2015-03-05 00:36:39,800 containers.py:669] Detected docker environment variables: DOCKER_HOST=tcp://192.168.59.103:2376, DOCKER_CERT_PATH=/Users/guilhermetorres/.boot2docker/certs/boot2docker-vm, DOCKER_TLS_VERIFY=1
And the complete stack trace:
DEBUG: Running gcloud.preview.app.run with Namespace(admin_host=None, allow_skipped_files=False, api_host=None, appidentity_email_address=None, appidentity_private_key_path=None, auth_domain='gmail.com', blobstore_path=None, clear_datastore=False, cmd_func=<bound method Command.Run of <googlecloudsdk.calliope.backend.Command object at 0x10e9e5b50>>, command_path=['gcloud', 'preview', 'app', 'run'], datastore_consistency_policy='time', datastore_path=None, default_gcs_bucket_name=None, enable_cloud_datastore=False, enable_mvm_logs=False, enable_sendmail=False, format=None, h=None, help=None, host=None, jvm_flag=None, log_http=None, log_level=None, logs_path=None, markdown=None, max_module_instances=None, modules=['./app.yaml'], php_executable_path=None, project=None, python_startup_script=None, quiet=None, require_indexes=False, show_mail_body=False, smtp_allow_tls=False, smtp_host=None, smtp_password=None, smtp_user=None, storage_path=None, trace_token=None, use_mtime_file_watcher=False, user_output_enabled=None, verbosity='debug', version=None).
WARNING: The [application] field is specified in file [/Users/guilhermetorres/meetapp-xmpp-docker/app.yaml]. This field is not used by gcloud and should be removed.
DEBUG: Found Cloud SDK root: /Users/guilhermetorres/google-cloud-sdk
DEBUG: Found App Engine SDK root: /Users/guilhermetorres/google-cloud-sdk/platform/google_appengine
DEBUG: Command sys.path: ['/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/antlr3', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/fancy_urllib', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/ipaddr', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/yaml-3.10', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/rsa', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/pyasn1', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/pyasn1_modules', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/httplib2', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/oauth2client', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/concurrent', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/cherrypy', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/distutils', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/requests', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/six', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/websocket', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/jinja2-2.6', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/webob-1.2.3', '/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/lib/webapp2-2.5.1', '/Users/guilhermetorres/google-cloud-sdk/./lib', '/Users/guilhermetorres/google-cloud-sdk/lib/googlecloudsdk/gcloud', '/Users/guilhermetorres/google-cloud-sdk/lib', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload']
DEBUG: Found Cloud SDK root: /Users/guilhermetorres/google-cloud-sdk
Module [default] found in file [/Users/guilhermetorres/meetapp-xmpp-docker/app.yaml]
INFO: Looking for the Dockerfile in /Users/guilhermetorres/meetapp-xmpp-docker
INFO: Using Dockerfile found in /Users/guilhermetorres/meetapp-xmpp-docker
DEBUG: Found Cloud SDK root: /Users/guilhermetorres/google-cloud-sdk
DEBUG: Skipping pull for runtime: custom
DEBUG: Running [dev_appserver.py] with: --allow_skipped_files=False --application=meetapp-xmpp --auth_domain=gmail.com --clear_datastore=False --datastore_consistency_policy=time --dev_appserver_log_level=debug --enable_cloud_datastore=False --enable_mvm_logs=False --enable_sendmail=False --log_level=debug --require_indexes=False --show_mail_body=False --skip_sdk_update_check=True --smtp_allow_tls=False --use_mtime_file_watcher=False /Users/guilhermetorres/meetapp-xmpp-docker/app.yaml
DEBUG 2015-03-05 00:36:39,759 application_configuration.py:159] setting forwarded ports 5280,5222,8088
INFO 2015-03-05 00:36:39,759 devappserver2.py:726] Skipping SDK update check.
DEBUG 2015-03-05 00:36:39,799 wsgi_server.py:384] Failed to bind "fe80::1%lo0:56987": ('Unable to bind fe80::1%lo0:56987', error(49, "Can't assign requested address"))
INFO 2015-03-05 00:36:39,799 api_server.py:172] Starting API server at: http://localhost:56987
DEBUG 2015-03-05 00:36:39,800 containers.py:669] Detected docker environment variables: DOCKER_HOST=tcp://192.168.59.103:2376, DOCKER_CERT_PATH=/Users/guilhermetorres/.boot2docker/certs/boot2docker-vm, DOCKER_TLS_VERIFY=1
ERROR 2015-03-05 00:36:39,816 containers.py:742] Failed to connect to Docker Daemon due to: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)
INFO 2015-03-05 00:36:39,817 api_server.py:588] Applying all pending transactions and saving the datastore
INFO 2015-03-05 00:36:39,817 api_server.py:591] Saving search indexes
Traceback (most recent call last):
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 83, in <module>
_run_file(__file__, globals())
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 79, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 985, in <module>
main()
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 978, in main
dev_server.start(options)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 774, in start
self._dispatcher.start(options.api_host, apis.port, request_data)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py", line 182, in start
_module, port = self._create_module(module_configuration, port)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py", line 262, in _create_module
threadsafe_override=threadsafe_override)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 1463, in __init__
super(ManualScalingModule, self).__init__(**kwargs)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 514, in __init__
self._module_configuration)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 237, in _create_instance_factory
module_configuration=module_configuration)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/vm_runtime_factory.py", line 78, in __init__
timeout=self.DOCKER_D_REQUEST_TIMEOUT_SECS)
File "/Users/guilhermetorres/google-cloud-sdk/platform/google_appengine/google/appengine/tools/docker/containers.py", line 743, in NewDockerClient
raise DockerDaemonConnectionError(DOCKER_CONNECTION_ERROR)
google.appengine.tools.docker.containers.DockerDaemonConnectionError: Couldn't connect to the docker daemon.
Please check if the environment variables DOCKER_HOST, DOCKER_CERT_PATH and DOCKER_TLS_VERIFY are set correctly. If you are using boot2docker, you can set them up by executing the commands that are shown by:
boot2docker shellinit
DEBUG: Error [1] while running DevAppSever with: [--allow_skipped_files=False --application=meetapp-xmpp --auth_domain=gmail.com --clear_datastore=False --datastore_consistency_policy=time --dev_appserver_log_level=debug --enable_cloud_datastore=False --enable_mvm_logs=False --enable_sendmail=False --log_level=debug --require_indexes=False --show_mail_body=False --skip_sdk_update_check=True --smtp_allow_tls=False --use_mtime_file_watcher=False /Users/guilhermetorres/meetapp-xmpp-docker/app.yaml]
DEBUG: (gcloud.preview.app.run) DevAppSever failed with error code [1]
Traceback (most recent call last):
File "/Users/guilhermetorres/google-cloud-sdk/./lib/googlecloudsdk/calliope/cli.py", line 551, in Execute
result = args.cmd_func(cli=self, args=args)
File "/Users/guilhermetorres/google-cloud-sdk/./lib/googlecloudsdk/calliope/backend.py", line 1025, in Run
result = command_instance.Run(args)
File "/Users/guilhermetorres/google-cloud-sdk/./lib/googlecloudsdk/calliope/exceptions.py", line 86, in TryFunc
return func(*args, **kwargs)
File "/Users/guilhermetorres/google-cloud-sdk/lib/googlecloudsdk/appengine/app_commands/run.py", line 317, in Run
runner.Start(*runnables)
File "/Users/guilhermetorres/google-cloud-sdk/./lib/googlecloudsdk/appengine/lib/appengine_adapter.py", line 413, in Start
raise DevappserverExecutionError(return_code, argv)
ToolException: DevAppSever failed with error code [1]
ERROR: (gcloud.preview.app.run) DevAppSever failed with error code [1]
After reading this issue https://github.com/docker/docker-py/issues/465, and uninstall homebrew (brew remove python)Python everything works fine.
Related
On release 8.1.1 I am trying to experiment with the simple web agent.
Running through the setup process
volttron -vv -l volttron.log --bind-web-address http://0.0.0.0:8080 &
Everything seem to install OK for http protrocol on the vcfg and starting the agent starts fine but going to the browser I get an empty page response.
And in terminal an error here's the Full traceback:
.do_close of <WSGIServer, (<gevent._socket3.socket [closed] at 0x7f64342242c)> failed with SSLError
Traceback (most recent call last):
File "src/gevent/greenlet.py", line 854, in gevent._gevent_cgreenlet.Greenlet.run
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/baseserver.py", line 34, in _handle_and_close_when_done
return handle(*args_tuple)
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/server.py", line 233, in wrap_socket_and_handle
with _closing_socket(self.wrap_socket(client_socket, **self.ssl_args)) as ssl_socket:
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 793, in wrap_socket
return SSLSocket(sock=sock, keyfile=keyfile, certfile=certfile,
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 311, in init
raise x
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 307, in init
self.do_handshake()
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 663, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: HTTP_REQUEST] http request (_ssl.c:1131)
2021-09-29T13:38:34Z <Greenlet at 0x7f64341fc480: _handle_and_close_when_done(<bound method StreamServer.wrap_socket_and_handle , <bound method StreamServer.do_close of <WSGIServer, (<gevent._socket3.socket [closed] at 0x7f643419195)> failed with SSLError
Traceback (most recent call last):
File "src/gevent/greenlet.py", line 854, in gevent._gevent_cgreenlet.Greenlet.run
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/baseserver.py", line 34, in _handle_and_close_when_done
return handle(*args_tuple)
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/server.py", line 233, in wrap_socket_and_handle
with _closing_socket(self.wrap_socket(client_socket, **self.ssl_args)) as ssl_socket:
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 793, in wrap_socket
return SSLSocket(sock=sock, keyfile=keyfile, certfile=certfile,
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 311, in init
raise x
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 307, in init
self.do_handshake()
File "/home/ben/Desktop/volttron/env/lib/python3.8/site-packages/gevent/_ssl3.py", line 663, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: HTTP_REQUEST] http request (_ssl.c:1131)
2021-09-29T13:38:34Z <Greenlet at 0x7f643423c6a0: _handle_and_close_when_done(<bound method StreamServer.wrap_socket_and_handle , <bound method StreamServer.do_close of <WSGIServer, (<gevent._socket3.socket [closed] at 0x7f64342242c)> failed with SSLError
EDIT
So if I do a nano ~/.volttron/config it looks like this below. I did change the bind-web-address for the IP address of my test bench instance. Hopefully that wasn't a mistake it looked like the initial bind-web-address was the name of the computer. --bind-web-address http://ben-hp-probook-6550b:8080
message-bus = zmq
vip-address = tcp://127.0.0.1:22916
instance-name = benshome
bind-web-address = http://192.168.0.105:8080
web-ssl-cert = /home/ben/.volttron/certificates/certs/platform_web-server.crt
web-ssl-key = /home/ben/.volttron/certificates/private/platform_web-server.pem
web-secret-key = 0e3b19770c0a8c0a08f274fcdabaf939fecc16601283266934c5ab258a1ed20cf440fde2c83cb8660dac569d31b5cdaf3ab7354a39b0640f355f9c5407c5fce619
I think I did first try HTTPS then resorted to HTTP. Anyways when I start VOLTTRON do I still need a --bind-web-address arg if the ~/.volttron/config is already setup with one?
I've a tried both when starting VOLTTRON to use the --bind flag or not but still unable to bring up a webpage on the IP address of the machine running VOLTTRON of 192.168.0.105. This would be the simple web agent, right?
I was able to reproduce this when I ran through vcfg and specified https, but then did what you did and passed the bind-web-address to the volttron command itself.
However, you shouldn't do this. The instructions assume you haven't gone through the vcfg process and therefore you would have to specify the bind web address on the command line.
Since you went through the vcfg process your config file (~/.volttron/config) will have your hostname:port as the bind-web-address. If it has https in it that is the reason it is not working for you.
Environment details:
Python 3.7.3
setuptools==51.0.0
I am just trying to install google-cloud-pubsub in coral developer board using the command,
pip install --upgrade --no-cache google-cloud-pubsub
When running the above command, it started installing the dependencies then after it goes for some grpcio build process as shown below.
Build log:
Building wheels for collected packages: grpc-google-iam-v1, grpcio
Building wheel for grpc-google-iam-v1 (setup.py) ... done
Created wheel for grpc-google-iam-v1: filename=grpc_google_iam_v1-0.12.3-py3-none-any.whl size=18501 sha256=2be51c55f5cab3ff97efdb39e3a6be07e5c2d030ca0a47a8358a8fa1efef58b6
Stored in directory: /tmp/pip-ephem-wheel-cache-rpj8jh5r/wheels/b9/ee/67/2e444183030cb8d31ce8b34cee34a7afdbd3ba5959ea846380
Building wheel for grpcio (setup.py) ... error
This process hangs for several minutes and got the error logs shown below printed.
Error:
Building wheel for grpcio (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/mendel/.virtualenvs/cv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-1pbjozx4/grpcio_3a1c8f81c7024b04bd7534fe601796ee/setup.py'"'"'; __file__='"'"'/tmp/pip-install-1pbjozx4/grpcio_3a1c8f81c7024b04bd7534fe601796ee/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-1u5g0n_z
cwd: /tmp/pip-install-1pbjozx4/grpcio_3a1c8f81c7024b04bd7534fe601796ee/
Complete output (1234 lines):
ASM Builds for BoringSSL currently not supported on: linux-aarch64
Found cython-generated files...
running bdist_wheel
......................
lot of lines there ...!
......................
Traceback (most recent call last):
File "/tmp/pip-install-1pbjozx4/grpcio_3a1c8f81c7024b04bd7534fe601796ee/src/python/grpcio/commands.py", line 267, in build_extensions
build_ext.build_ext.build_extensions(self)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions
self._build_extensions_serial()
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial
self.build_extension(ext)
File "/home/mendel/.virtualenvs/cv/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 196, in build_extension
_build_ext.build_extension(self, ext)
File "/usr/lib/python3.7/distutils/command/build_ext.py", line 534, in build_extension
depends=ext.depends)
File "/tmp/pip-install-1pbjozx4/grpcio_3a1c8f81c7024b04bd7534fe601796ee/src/python/grpcio/_parallel_compile_patch.py", line 59, in _parallel_compile
_compile_single_file, objects)
File "/usr/lib/python3.7/multiprocessing/pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
File "/usr/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "/tmp/pip-install-1pbjozx4/grpcio_3a1c8f81c7024b04bd7534fe601796ee/src/python/grpcio/_parallel_compile_patch.py", line 54, in _compile_single_file
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/usr/lib/python3.7/distutils/unixccompiler.py", line 120, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command 'aarch64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for grpcio
Running setup.py clean for grpcio
Note: I have successfully installed google-cloud-storage==1.33.0
I created a training script with hard coded input. It works as expected using a training job but I couldn't make it work using local mode.
It brings up a container on my local docker and exits with code (1)
Code:
estimator = SKLearn(entry_point="train_model.py",
train_instance_type="local")
estimator.fit()
Here is the exception:
2020-02-22 06:21:05,470 sagemaker-containers INFO Imported framework sagemaker_sklearn_container.training
2020-02-22 06:21:05,480 sagemaker-containers INFO No GPUs detected (normal if no gpus installed)
2020-02-22 06:21:05,504 sagemaker_sklearn_container.training INFO Invoking user training script.
2020-02-22 06:21:06,407 sagemaker-containers ERROR Reporting training FAILURE
2020-02-22 06:21:06,407 sagemaker-containers ERROR framework error:
Traceback (most recent call last):
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_trainer.py", line 81, in train
entrypoint()
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/training.py", line 36, in main
train(framework.training_env())
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/training.py", line 32, in train
training_environment.to_env_vars(), training_environment.module_name)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_modules.py", line 301, in run_module
_files.download_and_extract(uri, _env.code_dir)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_files.py", line 129, in download_and_extract
s3_download(uri, dst)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_files.py", line 164, in s3_download
s3.Bucket(bucket).download_file(key, dst)
File "/miniconda3/lib/python3.7/site-packages/boto3/s3/inject.py", line 246, in bucket_download_file
ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
File "/miniconda3/lib/python3.7/site-packages/boto3/s3/inject.py", line 172, in download_file
extra_args=ExtraArgs, callback=Callback)
File "/miniconda3/lib/python3.7/site-packages/boto3/s3/transfer.py", line 307, in download_file
future.result()
File "/miniconda3/lib/python3.7/site-packages/s3transfer/futures.py", line 106, in result
return self._coordinator.result()
File "/miniconda3/lib/python3.7/site-packages/s3transfer/futures.py", line 265, in result
raise self._exception
File "/miniconda3/lib/python3.7/site-packages/s3transfer/tasks.py", line 255, in _main
self._submit(transfer_future=transfer_future, **kwargs)
File "/miniconda3/lib/python3.7/site-packages/s3transfer/download.py", line 345, in _submit
**transfer_future.meta.call_args.extra_args
File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
An error occurred (403) when calling the HeadObject operation: Forbidden
tmpe_msr8pi_algo-1-kt1vh_1 exited with code 1
I found out that docker restart solved the issue.
After a while it happened again - and it solved it again.
I'm using docker for windows, and the issue is probably related to the created container configuration
Symptom:
I have a new standard environment endpoint framework (v2) project that works on the development server but when deployed to GCP the response is: This application is temporarily over its serving quota. Please try again later.
However if I try to add quota limits and metrics (as per https://cloud.google.com/endpoints/docs/frameworks/quotas-configure) the development server errors with: AttributeError: 'tuple' object has no attribute 'metric_name'.
The the full error logged is below. I suspect the warning about oauth2client is not relevant to the quota problem but is related to the current endpoints module.
The error seems to point to the limit_definitions but these are from the Google example.
Code snippet:
quota_limits = [
("read-requests", "Read Requests", 1000),
("list-requests", "List Requests", 100),
("write-requests", "Write Requests", 50),
]
#endpoints.api(
name='echo',
version='v1',
limit_definitions=quota_limits,
Question:
How can I either: not use quotas or correctly configure quotas? Or is there a bug?
SDK:
gcloud info
Google Cloud SDK [180.0.0]
Platform: [Linux, x86_64] ('Linux', 'host-name', '4.4.0-98-generic', '#121-Ubuntu SMP Tue Oct 10 14:24:03 UTC 2017', 'x86_64', 'x86_64')
Python Version: [2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]]
Python Location: [/usr/bin/python2]
Site Packages: [Disabled]
Installation Root: [/home/steve/google-cloud-sdk]
Installed Components:
core: [2017.11.10]
app-engine-python: [1.9.63]
gcloud: []
beta: [2017.09.15]
gsutil: [4.28]
cloud-datastore-emulator: [1.3.0]
bq: [2.0.27]
System PATH: [/home/steve/google-cloud-sdk/bin:/home/steve/.nvm/versions/node/v6.11.1/bin:/home/steve/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/lib/jvm/java-8-oracle/bin:/usr/lib/jvm/java-8-oracle/db/bin:/usr/lib/jvm/java-8-oracle/jre/bin:/home/steve/google-cloud-sdk/platform/google_appengine:/home/steve/Android/Sdk/tools:/home/steve/Android/Sdk/tools/bin:/home/steve/Android/Sdk/platform-tools:~/bin:~/.config/yarn/global/node_modules/.bin]
Python PATH: [/home/steve/google-cloud-sdk/lib/third_party:/home/steve/google-cloud-sdk/lib:/usr/lib/python2.7/:/usr/lib/python2.7/plat-x86_64-linux-gnu:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload]
Cloud SDK on PATH: [True]
Kubectl on PATH: [False]
There are alternate versions of the following Google Cloud Platform tools on your system PATH.
/home/steve/google-cloud-sdk/platform/google_appengine/dev_appserver.py
/home/steve/google-cloud-sdk/platform/google_appengine/endpointscfg.py
Installation Properties: [/home/steve/google-cloud-sdk/properties]
User Config Directory: [/home/steve/.config/gcloud]
Active Configuration Name: [default]
Active Configuration Path: [/home/steve/.config/gcloud/configurations/config_default]
Account: [<removed-for-public-post>]
Project: [project-name]
Current Properties:
[core]
project: [project-name]
account: [<removed for public post>]
disable_usage_reporting: [False]
Logs Directory: [/home/steve/.config/gcloud/logs]
Last Log File: [/home/steve/.config/gcloud/logs/2017.11.18/11.00.31.517561.log]
git: [git version 2.7.4]
ssh: [OpenSSH_7.2p2 Ubuntu-4ubuntu2.2, OpenSSL 1.0.2g 1 Mar 2016]
Full error displayed:
WARNING 2017-11-18 05:57:18,958 multistore_file.py:62] The oauth2client.contrib.multistore_file module has been deprecated and will be removed in the next release of oauth2client. Please migrate to multiprocess_file_storage.
ERROR 2017-11-18 05:57:19,080 wsgi.py:263]
Traceback (most recent call last):
File "/home/steve/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/home/steve/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/home/steve/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/home/steve/gae/prototyping/endpoint+fbauth/main.py", line 35, in <module>
'https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken#system.gserviceaccount.com')}
File "/home/steve/google-cloud-sdk/platform/google_appengine/lib/protorpc-1.0/protorpc/util.py", line 173, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/steve/gae/prototyping/endpoint+fbauth/lib/endpoints/api_config.py", line 1038, in api
limit_definitions=limit_definitions)
File "/home/steve/google-cloud-sdk/platform/google_appengine/lib/protorpc-1.0/protorpc/util.py", line 173, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/steve/gae/prototyping/endpoint+fbauth/lib/endpoints/api_config.py", line 508, in __init__
base_path=base_path, limit_definitions=limit_definitions)
File "/home/steve/google-cloud-sdk/platform/google_appengine/lib/protorpc-1.0/protorpc/util.py", line 173, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/steve/gae/prototyping/endpoint+fbauth/lib/endpoints/api_config.py", line 601, in __init__
_CheckLimitDefinitions(limit_definitions)
File "/home/steve/gae/prototyping/endpoint+fbauth/lib/endpoints/api_config.py", line 243, in _CheckLimitDefinitions
if not ld.metric_name:
AttributeError: 'tuple' object has no attribute 'metric_name'
quota_limits needs to be a list of endpoints.api_config.LimitDefinition instances, not a list of tuples. This is an error in the docs; I'll make sure they're corrected, and I'll improve the error message to be more clear.
I am getting the following error when trying to run the nosetest from my GAE project:
nosetests --nologcapture --with-gae --without-sandbox --gae-lib-root=/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine
but I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/nosetests", line 8, in <module>
load_entry_point('nose==1.3.4', 'console_scripts', 'nosetests')()
File "/Library/Python/2.7/site-packages/nose/core.py", line 121, in __init__
**extra_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/Library/Python/2.7/site-packages/nose/core.py", line 145, in parseArgs
self.config.configure(argv, doc=self.usage())
File "/Library/Python/2.7/site-packages/nose/config.py", line 346, in configure
self.plugins.configure(options, self)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 284, in configure
cfg(options, config)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 99, in __call__
return self.call(*arg, **kw)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 167, in simple
result = meth(*arg, **kw)
File "/Library/Python/2.7/site-packages/nosegae.py", line 87, in configure
from google.appengine.tools import old_dev_appserver as dev_appserver
ImportError: cannot import name old_dev_appserver
The sys.path reads:
'/Users/dsinha/Downloads/eclipse/plugins/org.python.pydev_3.9.0.201411111611/pysrc', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/antlr3', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/fancy_urllib', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/ipaddr', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/yaml-3.10', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/rsa', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1_modules', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/simplejson', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/django-1.4', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/goo...
so the app engine libraries should be getting pulled in. Actually its first failing to pull in dev_appserver, and then trying and failing to pull old_dev_appserver
The dir inside the appengine/lib file is:
bash-3.2$ ls
__init__.py appcfg.py backends_xml_parser.py dispatch_xml_parser.py handler_generator.py php_cli.py value_mixin.pyc
__init__.pyc appcfg_java.py boolean_action.py docker handler_generator.pyc queue_xml_parser.py web_xml_parser.py
adaptive_thread_pool.py appengine_rpc.py boolean_action.pyc dos_xml_parser.py jarfile.py queue_xml_parser.pyc web_xml_parser.pyc
api_server.py appengine_rpc.pyc bulkload_client.py download_appstats.py java_quickstart.py remote_api_shell.py xml_parser_utils.py
app_engine_config_exception.py appengine_rpc_httplib2.py bulkloader.py endpointscfg.py java_quickstart.pyc requeue.py xml_parser_utils.pyc
app_engine_config_exception.pyc augment_mimetypes.py cron_xml_parser.py gen_protorpc.py java_utils.py sdk_update_checker.py yaml_translator.py
app_engine_web_xml_parser.py augment_mimetypes.pyc dev-channel-js.js handler.py java_utils.pyc sdk_update_checker.pyc yaml_translator.pyc
app_engine_web_xml_parser.pyc backends_conversion.py devappserver2 handler.pyc os_compat.py value_mixin.py
bash-3.2$ pwd
/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/google/appengine/tools
I also tried to find the modules available inside the google.appengine.tools package:
>>> import pkgutil
>>> [name for _, name, _ in pkgutil.iter_modules(['testpkg'])]
[]
This problem started occuring after I upgraded to App Engine 1.9.10 (to use the async search features). In a problem that I think is related, when I try to run the debug server from PyDev, it just silently terminated on any page request (localhost:8080).
Running dev_appserver . from the command line works fine though.
Nose-GAE broke with App Engine 1.9.17: https://github.com/Trii/NoseGAE/issues/6
Downgrading to 1.9.15 made the problem go away temporarily while waiting for the issue to be resolved by nose-gae