How to run managed-vm-gae example code locally - google-app-engine

I followed this tutorial
to get a Bigtable client up and running in Google Managed VMs. But is there a way to run this locally? Reason is that deploying the code remotely in development is a pain.
Normally I can use dev_appserver.sh to run GAE app locally. But when I run it, I'm getting this error:
Caused by: java.lang.IllegalStateException: Jetty ALPN has not been
properly configured.
Which means we need to include ALPN library? Since our codebase is in Java 7, I used this ALPN version: 7.1.3.v20150130.
I then tried again with this:
dev_appserver.sh --jvm_flag=-Xbootclasspath/p:/Users/shouguoli/tmp/alpn-boot-7.1.3.v20150130.jar
still getting this error:
Caused by: com.google.apphosting.api.ApiProxy$CallNotFoundException:
The API package 'urlfetch' or call 'Fetch()' was not found.
How do you get it to work locally?

The sample was updated last week. It's based on the java 8 compat runtime, which means that you have access to most of the App Engine API's including Users, Task Queues, and Datastore.
There is a new Netty TCNative module that uses Boring SSL.
To use it with the pom.xml in the sample, do:
mvn clean -Pmac jetty:run -Dbigtable.projectID=<your-project> -Dbigtable.clusterID=<your-cluster> -Dbigtable.zone=<your-zone>
To use on Windows, use -Pwindows instead of -Pmac. For linux, omit the Profile -P as it's the default.
To deploy:
mvn clean gcloud:deploy -Dbigtable.projectID=<your-project> -Dbigtable.clusterID=<your-cluster> -Dbigtable.zone=<your-zone>
NOTE - it is advisable to do the clean between running locally and running remotely as the TCNative module is currently specific to the platform the code runs on.
We are in the process of updating all of our samples to use TCNative, we hope to have this by 3/10/16.

Related

Django+React AWS LightSails Ubuntu deployment not using js

this is my first deployment, I have gone so far as to create and configure Ubuntu instance on lightsails following official guide. However, I run into problems when instead of an empty project I use my own Django hosting React frontend.
My project works on my computer and if I start up django's production server thorugh manage.py runserver but does not run at all if I use gunicorn to run it with:
gunicorn my_app.wsgi --bind 0.0.0.0:8000 --workers 4
It seems that it cannot find proper filepaths of any static files but I do not undertstand how to configure it to use it since Django already has all the right paths.
One of the errors in the browser console:
Refused to execute http://3.127.76.103/static/rest_framework/js/bootstrap.min.js as script because "X-Content-Type-Options: nosniff" was given and its Content-Type is not a script MIME type.
How can I start fixing it, I do not understand why it does not see the any files.

Deploying Haskell yesod docker container on google app engine

I am trying to upload a yesod Docker container on Google App Engine. The source code is here and the Docker image is here.
I followed the documentation in the Custom runtime quickstart, and when invoking gcloud app deploy the app builds fine after increasing the build timeout, but the container either the readiness check when trying to start or shows the following timeout message:
ERROR: (gcloud.app.deploy) Operation [apps/meeshkan-github-webhook-router/operations/xxxx-xxxx-xxxx] timed out. This operation may still be underway.
I have tried experimenting with several things, including a manual readiness check, creating an /_ah/health endpoint, and increasing the timeout of the readiness check all the way to 1799 seconds, but none of these actions seem to work.
One issue may be the size of the container (it is 3.2gb), and I could try to prune it down, but I'd only do that if someone could confirm that container size is a contributing factor to deployment problems. Other than that, I'm not sure what could be causing this failure. The docker image starts fine on our local machines.
Thanks in advance for your help and suggestions!
The issue turned out to be that, because I was building on Windows, images built using Docker Desktop on Windows gave all shell scripts executable permission automatically, whereas Docker on Linux needs shell scripts to be given the executable permission. By adding this line to my Dockerfile:
RUN chmod +x /usr/src/app/run.sh
Everything worked fine!

Go app engine dev_appserver.py doesn't support runtime go112

I am trying to migrate my Go app to runtime go112, but dev_appserver.py throw the following error:
RuntimeError: Unknown runtime 'go112'; supported runtimes are 'custom', 'go', 'go111', 'java', 'java7', 'java8', 'php55', 'php72', 'python', 'python-compat', 'python27', 'python37'.
Current Go version is 1.12.11
Edit: I used apt-get to install the SDK.
apt-get update && apt-get install google-cloud-sdk google-cloud-sdk-app-engine-python\
google-cloud-sdk-app-engine-python-extras\
google-cloud-sdk-app-engine-java\
google-cloud-sdk-app-engine-go\
google-cloud-sdk-datalab\
google-cloud-sdk-datastore-emulator\
google-cloud-sdk-pubsub-emulator\
google-cloud-sdk-cbt\
google-cloud-sdk-cloud-build-local\
google-cloud-sdk-bigtable-emulator\
kubectl -y
That way it seems I can't use gcloud components update and the packages are not up-to-date in the repos.
Now my questions is: Does google-cloud-sdk-app-engine-go support go runtime 1.12? If not, when will it support runtime 1.12?
Update: I installed cloud SDK and run gcloud components update successfully on another computer, dev_server.py still throw the same error. Does the local SDK actually support rumtime 1.12?
go112 runtime is not available on local dev_appserver.py. This has been discussed on the App Engine Go forum at
https://groups.google.com/forum/#!topic/google-appengine-go/kHxZ9zIb_QE
The expectation is that starting from go112 runtime, there is no "appengine" way of running the go server and works normally like any go server outside GAE. There are a few open issues like dispatching urls though.
It sounds like you gcloud SDK is an old version and not recognize "runtime: go112", please run this command in order to update your SDK.
gcloud components update
After the update you can be able to deploy an app engine service using GO 1.12 runtime, check the changes that you need to perform in you App.yaml in this link

"ImportError: No module named _ssl" with dev_appserver.py from Google App Engine

Background
"In the Python runtime, we've added support for the Python SSL
Library, so you can now open secure connections to remote services
such as Apple's Push Notification service."
This quote is taken from a recent post on the Google App Engine blog.
Implementation
If you want to use native python ssl, you must enable it using the libraries configuration in your application's app.yaml file where you specify the library name "ssl" . . .
These instructions are provided for developers through the Google App Engine documentation.
The following lines have been added to the app.yaml file:
libraries:
- name: ssl
version: latest
This much is in line with the advice provided through the Google App Engine documentation.
Problem
I have tried running my project in three different configurations. Two are working, and one is not.
Working ...
After I upload my application to Google App Engine, and run my project through the live server, everything works fine.
Working ...
When I run my project with manage.py runserver and include the Google App Engine SKD in my PYTHONPATH, everything works fine.
Not Working ...
However, when I run my project with dev_appserver.py, I get the following error:
ImportError at /
No module named _ssl
Request Method: GET
Request URL: http://localhost:8080/
Django Version: 1.4.3
Exception Type: ImportError
Exception Value:
No module named _ssl
Exception Location: /usr/local/lib/google_appengine_1.7.7/google/appengine/tools/devappserver2/python/sandbox.py in load_module, line 856
Python Executable: /home/rbose85/Code/venvs/appserver/bin/python
Python Version: 2.7.3
Python Path:
['/home/rbose85/Code/product/site',
'/usr/local/lib/google_appengine_1.7.7',
'/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
'/usr/local/lib/google_appengine_1.7.7',
'/usr/local/lib/google_appengine_1.7.7',
'/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
'/usr/local/lib/google_appengine_1.7.7',
'/usr/local/lib/google_appengine_1.7.7/lib/protorpc',
'/home/rbose85/Code/venvs/appserver/lib/python2.7',
'/home/rbose85/Code/venvs/appserver/lib/python2.7/lib-dynload',
'/usr/lib/python2.7',
'/usr/local/lib/google_appengine',
u'/usr/local/lib/google_appengine_1.7.7/lib/django-1.4',
u'/usr/local/lib/google_appengine_1.7.7/lib/ssl-2.7',
u'/usr/local/lib/google_appengine_1.7.7/lib/webapp2-2.3',
u'/usr/local/lib/google_appengine_1.7.7/lib/webob-1.1.1',
u'/usr/local/lib/google_appengine_1.7.7/lib/yaml-3.10']
Server time: Wed, 24 Apr 2013 11:23:49 +0000
For the current GAE version (1.8.0 at least until 1.8.3), if you want to be able to debug SSL connections in your development environment, you will need to tweak a little bit the gae sandbox:
add "_ssl" and "_socket" keys to the dictionary _WHITE_LIST_C_MODULES in /path-to-gae-sdk/google/appengine/tools/devappserver2/python/sandbox.py
Replace the socket.py file provided by google in /path-to-gae-sdk/google/appengine/dis27 from the socket.py file from your Python framework.
IMPORTANT: Tweaking the sandbox environment might end up with functionality working on your local machine but not in production (for example, GAE only supports outbound sockets in production). I will recommend you to restore your sandbox when you are done developing that specific part of your app.
The solution by jmg works, but instead of changing the sdk files, you could monkey patch the relevant modules.
Just put something like this on the beginning of your project setup.
# Just taking flask as an example
app = Flask('myapp')
if environment == 'DEV':
import sys
from google.appengine.tools.devappserver2.python import sandbox
sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
from lib import copy_of_stdlib_socket.py as patched_socket
sys.modules['socket'] = patched_socket
socket = patched_socket
I had to use a slightly different approach to get this working in CircleCI (unsure what peculiarity about their venv config caused this):
appengine_config.py
import os
if os.environ.get('SERVER_SOFTWARE', '').startswith('Development'):
import imp
import os.path
import inspect
from google.appengine.tools.devappserver2.python import sandbox
sandbox._WHITE_LIST_C_MODULES += ['_ssl', '_socket']
# Use the system socket.
real_os_src_path = os.path.realpath(inspect.getsourcefile(os))
psocket = os.path.join(os.path.dirname(real_os_src_path), 'socket.py')
imp.load_source('socket', psocket)
I had this problem because I wasn't vendoring ssl in my app.yaml file. I know the OP did that, but for those landing here for the OP's error, it's worth making sure lines like the following are in your app.yaml file:
libraries:
- name: ssl
version: latest
Stumbled upon this thread trying to work with Apples Push notification service and appengine... I was able to get this working without any monkey patching, by adding the SSL library in my app.yaml, as recommended in the official docs, hope that helps someone else :)
I added the code to appengine_config.py as listed by Spain Train, but had to also add the following code as well to get this to work:
phttplib = os.path.join(os.path.dirname(real_os_src_path), 'httplib.py')
imp.load_source('httplib', phttplib)
You can test if ssl is available at your local system by opening a python shell and typing import ssl. If no error appears then the problem is something else, otherwise you don't have the relevant libraries installed on your system. If you are using a Linux operating system try sudo apt-get install openssl openssl-devel or the relevant instructions for your operating system to install them locally. If you are using windows, these are the instructions.

apache-cxf client works with exec-maven-plugin, but not from outside

I am having problems triying to create a WS client in java.
The libraries I'm using is apache-cxf 2.12. This is an old version but I want the web service to be added to a JBoss application that is already running and uses Spring 2.5.6, so this org.apache.cxf version uses same spring version.
But the question is (I know is quite generic, is related to maven, netbeans and apache-cxf)
When I execute my client project (maven proyect) from netbeans, it works sort of Ok.
When I try to run this class with a script (bot windows an linux) it gives me this error:
"Invocation failed with the following: org.apache.cxf.ws.policy.PolicyException: None of the policy alternatives can be satisfied."
Java versions are the same, I've copied all the dependency jar into lib dir and added then to %classpath%
So, what can be different that when I execute my sample program with maven org.codehaus.mojo:exec-maven-plugin:1.2.1:exec it works and it doesn't when I run it with a srcipt? (I have checked all dependencies, java version, ... )
I guess it has to be related to JAX-WS o JCE (Java Cryptography Architecture) initialization.
¿Is there a way to see what %classpath maven is creating when in runs java.exe?
Maven script that works:
JAVA_HOME=D:\\LOCAL\\Java\\jdk1.5.0_15
D:\\LOCAL\\apache-maven-2.0.9\\bin\\mvn.bat
-Dexec.classpathScope=runtime
-DskipTests=true
"-Dexec.args=-Djavax.net.debug=all -classpath %classpath es.webservice.aaTest TESTFILE"
-Dexec.executable=D:\\LOCAL\\Java\\jdk1.5.0_15\\bin\\java.exe
-Dmaven.repo.local=D:\\bsrepo\\.m2 process-classes
org.codehaus.mojo:exec-maven-plugin:1.2:exec
Thanks

Resources