CouldNotFindModuleError on Google App Engine and Python 2.7 - google-app-engine

There is another question on this but i didn't found it useful.
I've installed Google App Engine SDK 1.7 with Python 2.7 on Windows 7.
When i launch the app from the Google App Engine Launcher, i have this error in the logs
CouldNotFindModuleError
the entire log is here:
http://pastebin.com/EwbpMcx2
ok it could not find a module, but what module? how can i solve this?
my app is very simple, a hello world
app.yaml
application: alleudacity
version: 1
runtime: python27
api_version: 1
threadsafe: no
handlers:
- url: /.*
script: helloworld.py
helloword.py
print 'Content-Type: text/plain'
print ''
print 'Hello, world!'

It looks like you're using libraries elixir and paramiko which are not available by default on google app engine.
You can either not use them (good way to test if that's really the problem), or potentially if they are pure python libraries, install them in your actual project folder.
When you deploy to app engine, only files in your project folders are uploaded - libraries that installed elsewhere in your system won't be available. dev_appserver modifies your environment so that you see these failures locally, rather than having things work fine locally and break after you deploy.

I guess I misread the trace. You're missing some module. Looking at the google code, it should print out the name of the module you're missing. But since I don't see it, you could force a breakpoint right before where the error should be and see what the name of the missing module is. Force a breakpoint by editing the code (C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_import_hook.py) and adding
import pdb
pdb.set_trace()

Related

How to deploy a Helidon application to Google Cloud App Engine?

I am trying to deploy an Helidon MP project to Google Cloud App Engine using java11 runtime but having trouble to define the app.yaml properly.
Tried to deploy the jar file directly using the below app.yaml using the command $ gcloud app deploy cord.jar. The app gets deployed but empty page on view.
runtime: java11
entrypoint: 'java -jar cord.jar'
Tried to modify the codewbase adding appengine\app.yaml to <project>\src\main\appengine\app.yaml and with contents as below and using command $ gcloud app deploy pom.xml:
runtime: java11
instance_class: F1
In all cases, the app got deployed but page loads empty.
They have examples on github but unfortunately not yet with Helidon.
I've put together an example for Helidon.
A couple things to note:
Make sure your application obeys the PORT environment variable, and configures its server to use that port.
Make sure your app.yaml is in the same directory as your jar and defines a custom entry point. For example:
runtime: java11
entrypoint: java -Xmx64m -jar helidon-quickstart-se.jar
Helidon uses "thin" jars and App Engine seems to handle this AOK as mentioned here: https://cloud.google.com/appengine/docs/standard/java11/runtime#application_startup
As an answer to my question.. issue for the page not loading was because of port 9090 that we were using (defined in src/main/resources/META-INF/microprofile-config.properties file). After i changed it to default 8080, my app worked.
microprofile-config.properties:
# Application properties. This is the default greeting
app.greeting=Hello
# Microprofile server properties
server.port=8080
server.host=0.0.0.0
References:
Helidon MP example for Google App Engine
There is a github thread regarding this and so far the current workaround is to add an app.yaml file similar to the one for the frameworks like Spring Boot or Vert.x
I have followed the tutorial where the github sample of the other responses is and it worked for me.
First I have cloned the repository and I used the quickstart mp:
git clone https://github.com/barchetta/helidon-google-app-engine-example/
cd helidon-google-app-engine-example/helidon-quickstart-mp
Then I have built and run the application and check if the port responds.
mvn package
export PORT=8888
java -jar target/helidon-quickstart-mp.jar
After all these previous steps I was able to see in localhost the result of the application.
For deploying I created the app.yaml file named "helidon-mp-app.yaml" and wrote this configuration inside:
runtime: java11
entrypoint: java -Xmx64m -jar helidon-quickstart-mp.jar
And copied it to the target/ directory:
cp helidon-mp-app.yaml target/
As the last configuration file, the file ".gcloudingonre" which also needs to be moved to target/
# Exclude everything. Then include just the app jar and runtime
# dependencies in libs/
*
*/
*/**
!helidon-quickstart-mp.jar
!libs/
!libs/**
Then as all the configuration files are ready, I executed
gcloud app deploy target/helidon-mp-app.yaml
gcloud app browse
And appending "/greet" in the URL we can see the result:
{"message":"Hello World!"}

App Engine not showing static file directory in Debugger Source view

I inherited a App Engine static-file-only service and I am still trying to understand it (contract developers, long gone, docs=0). One point that I am stuck on is why in the Debugger Source display I don't see the static file directory. This is a Python 2.7 app and to be clear there is no Python code provided. The app.yaml file looks like this:
service ya-da
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
static_files: dist/index.html
upload: dist/index.html
It is a SPA, so it just one html file and lots of bundled JS & CSS.
This is what I see in App Engine Debugger source display
So no dist directory is shown. But it is there as the app is running happily. There is nothing tucked into any other directory (most of them are utterly useless in the App Engine context) that would be the contents of dist.
So, the simple question is: why doesn't dist show up in the debugger?
given that you have found an issue related with the debugger, you can file an issue in the Public Issue Tracker of Google Cloud Platform with the components of Debugger Stackdriver here. In this way, the Stackdriver Engineers team have visibility of this issue and other users with the same issue can star it and follow it.
From your comment to my other answer, it seems you just need access to code that was deployed.
It looks like you are using GAE standard first generation so you should be able to download your code as described here. This is the command:
appcfg.py -A [YOUR_PROJECT_ID] -V [YOUR_VERSION_ID] download_app [OUTPUT_DIR]
That page has deprecation warnings because Google is encouraging people to migrate to GAE second generation, but I expect that will work for you.
Your static files don't show up in the debugger because GAE treats static files very differently than it does your code.
The point of having static files is to be able to serve them without burdening your GAE instances. The app.yaml file has a an upload option because you are telling GAE where to put your static files, and this location is separate from where your code is uploaded.
The debugger is for debugging code. It is not possible to debug a static file because there is no code (Well, no server code. Your Javascript runs in the browser.) and there is nothing to debug.
What is it you are trying to accomplish? Your Javascript is running on the front end so you can't debug it on the backend.

Dependencies (requirements.txt) not installing correctly when deploying to Google App Engine using Google Cloud Build

I am experiencing a strange issue with dependencies when deploying my application to Google App Engine (Python 2.7).
I have specified my dependencies in the requirements.txt file:
Flask==1.0.2
werkzeug>=0.14
flask-cors
twilio
httplib2
gunicorn
Jinja2
google-cloud-ndb
exponent_server_sdk
These work fine with the local development server when installed locally with pip from the requirements.txt folder.
However, when I deploy to app engine, it appears that the module exponent_server_sdk has not been installed (ImportError: No module named exponent_server_sdk).
I normally deploy by pushing to a git repository which is then deployed with Google Cloud Build ("gcr.io/cloud-builders/gcloud"). There are no errors when I deploy and other dependencies in the requirements.txt file work fine such as twilio or Jinja2.
Whilst investigating this I tried pushing directly from my development machine with gcloud app deploy. When I do this, for some strange reason, exponent_server_sdk is available and works correctly.
This behaviour seems very strange to me and I can't find any documentation of people experiencing similar errors. I wonder if anyone can give me any guidance about what might be causing this issue or places I can look to find errors (for example logs of the process of installing the requirements.txt file during deployment/instance startup).
In response to the comments:
I am running in the standard environment.
My cloudbuild.yaml looks like this:
steps:
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "app.yaml", "dispatch.yaml", "service_1.yaml", "service_2.yaml", "service_3.yaml", "index.yaml"]
timeout: "1600s"
The .yaml file for the service in which I am experiencing the error looks like this:
runtime: python27
api_version: 1
threadsafe: true
service: notifications
libraries:
- name: ssl
version: latest
handlers:
- url: /.*
script: notifications.app
For the (1st gen) standard environment the deployment process does not use a requirements.txt file, the libraries should be installed inside your app. From Copying a third-party library:
Use pip (version 6 or later) with the -t <directory> flag to
copy the libraries into the folder you created in the previous step.
For example:
pip install -t lib/ <library_name>
Yes, if you want you can use a requirements.txt file to do the installation inside your app, but that file itself is not used by the deployment process.
Since you're not deploying directly from the environment in which you had the libraries installed you'd have to add the installation directory to the git repo as well.

Can I have application versions in python27 and python37 in Google App Engine Standard (both types)

I want to test Python 3.7 on Google App Engine Standard.
I am afraid if I upload application with python37 old application will stop working and will be no revert to previous python27.
Can you share some experience if it possible to keep both runtime versions?
It is not clear for me.
I read Python 3.7 on Google App Engine - documentation but not found information.
You have 3 main options, ranked from easier to 'harder' to do:
Just deploy this test you want to do in a new project, and you don't have to worry about your old application getting replaced.
Deploy this test to a new service. Just add the line service: whatevername to the app.yaml file of your test, and deploy it as usual with $gcloud app deploy. Voilà, you have a new service.
You can deploy this test in your default service, but to avoid stopping the previous one, add the --no-stop-previous-version flag to your $ gcloud app deploy command. After that, you can split traffic between this test and your actual app.
To be honest, I would go with options 1 or 2, as they are far more simple to deploy, and you can later on delete the test project or delete the new service if you wish to.

Golang Cloud SDK - gcloud app deploy cannot find import package

according to the official documentation for Google App Engine Standard environment (Go API) the "preferred tooling to deploy a project" is now the Cloud SDK and so we moved to gcloud from goapp.
We are unable to deploy Go projects to GAE because all the sub-packages of every given project can't be found at "deploy time".
The typical folder structure that we have been using for every GAE project was as follows:
-project-name
--app.yaml
--main.go
--assets
---package1
---package2
When global libraries were put in the system GOPATH everything worked smoothly.
Running gcloud app deploy we now get this:
You are about to deploy the following services:
- yourproject/default/123456789 (from [/Path/to/app.yaml])
Deploying to URL: [https://yourproject.appspot.com]
Do you want to continue (Y/n)? Y
Beginning deployment of service [default]...
ERROR: (gcloud.app.deploy) Staging command [/path/to/yourproject/app.yaml /var/folders/b6/5ydn0wdn64jd32sxzzz48b7c0000gn/T/tmpbd4oiG] failed with return code [1].
------------------------------------ STDOUT ------------------------------------
------------------------------------ STDERR ------------------------------------
2017/03/24 10:25:58 failed analyzing /path/to/yourproject: cannot find package "yourpackage" in any of:
($GOROOT not set)
/path/to/gopath/src/yourpackage (from $GOPATH)
GOPATH: /path/to/gopath
--------------------------------------------------------------------------------
while dev_appserver.py works perfectly keeping the same folder structure.
Did we miss something?
How can we deploy to Google App Engine Standard environment using gcloud?
If the project structure needs to be changed: how? Is there official documentation about it?
Thanks in advance,
Edit -- Further infos:
Luigi-Mac-Pro:path/to/yourproject distudio$ gcloud version
Google Cloud SDK 148.0.0
app-engine-go
app-engine-go-darwin-x86_64 1.9.50
app-engine-python 1.9.50
bq 2.0.24
bq-nix 2.0.24
core 2017.03.17
core-nix 2016.11.07
gcloud
gcloud-deps 2017.03.17
gcloud-deps-darwin-x86_64 2017.02.21
gsutil 4.23
gsutil-nix 4.22
Google recommends keeping your dependencies outside of the app directory, and using GOPATH to refer them. In your case, that would mean doing the following:
-project-name
--app.yaml
--main.go
where main.go contains
import (
"package1"
"package2"
)
And somewhere else:
-my_packages
--src
---package1
---package2
And then set GOPATH environment variable to path/to/my_packages prior to running dev_appserver and gcloud app deploy.
For the future
We are working out the long term solution for properly vendoring packages inside your app directory – likely using Go's future native package manager. I'm sorry to say that we don't have a good way of supporting sub-packages for gcloud app deploy. This was an unfortunate side effect of compatibility with the App Engine Flexible environment.

Resources