I am building a RESTful API using Python 3.6, the Falcon Framework, Google App Engine, and Firebase Cloud Firestore. At runtime I am receiving the following error ...
File "E:\Bill\Documents\GitHubProjects\LetsHang-BackEnd\lib\google\cloud\firestore_v1beta1\_helpers.py", line 24, in <module> import grpc
File "E:\Bill\Documents\GitHubProjects\LetsHang-BackEnd\lib\grpc\__init__.py", line 22, in <module>
from grpc._cython import cygrpc as _cygrpc
ImportError: cannot import name cygrpc
When researching StackOverFlow, I found an article regarding an AWS Lambda deployment, but it suggests a solution based on Docker. Docker is not a viable solution for us. I also found an article off StackOverflow that suggests running "pip install grpcio". We did not without luck.
We build the App Engine dependencies using a requirements.txt file. This file has the following contents ...
falcon==1.4.1
google-api-python-client
google-cloud-firestore
firebase-admin
enum34
grpcio
We apply the requirements file using the command ...
pip install -t lib -r requirements.txt
The App Engine server is started with the command ...
dev_appserver.py .
The development environment is Windows 10.
You seem to be mixing up the GAE standard and flexible environments:
using Python 3.6 is only possible in the flexible environment (which, BTW, is fundamentally Docker-based)
installing app dependencies in the lib directory and using dev_appserver.py for local development are only applicable to the standard environment
Somehow related: How to tell if a Google App Engine documentation page applies to the standard or the flexible environment
Ok. I will write up my findings just in case there's another fool like me.
First, Dan's response is correct. I was mixing standard and flexible environments. I had looked up a method for using the Falcon Framework with App Engine; as it turns out, the only article uses the standard environment. So that's how I wound up using dev_appserver.py. My app, however, is Python 3.6 and has dependencies that prevent stepping down to 2.7.
To develop locally for the flexible environment, you simply need to run as you normally would. In the case of Falcon Framework, that means using the Waitress wsgi server.
I find that it is a good practice to build and use a Python virtual environment. You use the virtualenv command for that. At deployment time, Google builds a docker container for the app in the cloud. To reconstruct all the necessary Python packages, you have to supply a requirements.txt file. If you have a virtual environment, then the requirements file is easily produced using pip freeze.
Related
Recently tried to update my Gaelyk project (yes, it's old, but it works well and I still use it), but Google App Engine will no longer accept the update. The error message returned is "Deployments using appcfg are no longer supported. See https://cloud.google.com/appengine/docs/deprecations". The thing is, I never used appcfg to deploy my application; I used Gaelyk and Gradle. But obviously Gaelyk must have used appcfg under the covers.
I did download the replacement Google Cloud SDK, but this new tool is not similar at all to how Gaelyk and Gradle worked. Is there anything I can do to get Gaelyk to work anymore? Or is Gaelyk just dead and I need to rewrite my application (like in Node.js or something instead of Groovy).
This will be hard, however I will try to help you as possible. I think you may try to migrate it somehow to app.yaml configuration of GAE.
I am not sure what plugins are used in the project. From Gaelyk temple project I can see that it's using appengine-geb which, according to the documentation, behind the scenes, is using gradle-appengine-plugin (there is wrong link on this doc, but proper is bellow).
On the github of gradle-appengine-plugin I have found following.
There is a note:
NOTE: All App Engine users are encouraged to transition to the new
gradle plugin for their projects.
And in FAQ part there is following information:
How do I deploy with gcloud?
If you're using gcloud to deploy your application, the newest version of app deploy > doesn't support war
directories, you will need to provide it with an app.yaml OR you can
use the appengineStage task to create a directory that is deployable
in /build/staged-app
$ ./gradlew appengineStage
$ gcloud app deploy build/staged-app/app.yaml --project [app id]
--version [some version]
NOTES:
You must explicitly define all config files your want to upload
(cron.yaml, etc)
This does not work with EAR formatted projects.
I think the best option will be to migrate to new appenine plugin or if not possible try to implement is with gcloud app deploy command crating the config files manually (at least app.yaml). And for this migration I can provide you this document.
I hope you will manage somehow...
I can confirm that Serge's answer on the Gaelyk Groups site works; the same procedure that he figured out also worked for me. To summarize:
Run gradlew appengineRun as run previously with Gaelyk.
Copy all jar files inside the build\exploded-app\WEB-INF\lib folder into a \src\main\webapp\web-inf\lib folder (for me the new lib folder did not exist previously).
To deploy, use the new required gcloud tool, and instead of running gradlew appengineUpdate (which fails now), instead run
gcloud app deploy appengine-web.xml where that XML file can be found in your webapp/WEB-INF directory. I navigated to that directory to run the gcloud command, but you can use a relative path there if your working directory is elsewhere. (There are a number of optional flags associated with the gcloud app deploy command, but I didn't need any of them.)
Serge needed to use these instructions to convert datastore-indexes.xml to index.yaml and run gcloud app deploy index.yaml, however, I didn't need to do this because I had no datastores.
I'm testing Google App Engine and trying to run a simple function to upload files to either the Blobstore or Cloud Storage. I'm typing the Python code directly in the Cloud Shell of my instance. The code is failing when I call:
from google.appengine.ext import blobstore
I get the error code:
Traceback (most recent call last):
File "upload_test.py", line 1, in <module>
from google.appengine.api import users
ImportError: No module named 'google.appengine'
Even though the documentation says that: You can use Google Cloud Shell, which comes with git and Cloud SDK already installed, I've tried installing a bunch of libraries:
gcloud components install app-engine-python
pip install google-cloud-datastore
pip install google-cloud-storage
pip install --upgrade google-api-python-client
I'm still getting the same error. How can I get the appengine library to work? Alternatively, is this the wrong method for creating an app that allows the user to upload files?
The google.appengine module is baked into the first-generation Python (2.7) runtime. It's not available to install via pip, in the second-generation (3.7) runtime, or in Cloud Shell.
The only way to use it is by writing and deploying a first-generation App Engine app.
Thanks #Dustin Ingram
I found the answer in this page.
The current "correct" way of uploading to Cloud Storage is to use google.cloud.storage. The tutorial I linked above explains how to implement it.
The impression I have, however, is that this uses twice the bandwidth as the solution via google.appengine. Originally, the front-end would receive an upload url and send the file directly to the Blobstore (or to Cloud Storage). Now the application uploads to the back-end which, in turn, uploads to Cloud Storage.
I'm not too worried, as I will not be dealing with excessively large files, but it seems strange that the ability to upload directly has been discontinued.
In any case, my problem has been solved.
I have an app that uses the following command to deploy a static app generated by create-react-app:
gcloud app deploy --project MY_PROJECT -v dev
After that, my app is available on myproject.appstop.com.
But I don't figure out how to deploy this app using a diff URL for each environment. Like.: dev.myproject.appspot.com, stg.myproject.com and so on.
If you know or have other ideas about how to solve this please share your thoughts.
It depends on what you understand by environment. I would definitely recommend you to create different projects for the different environments (my-project-dev, my-project-staging, my-project-test, my-project-prod...)
If you just want to have a different url for your deployment, you are doing ok by using the -v (version parameter).
Once you've deployed again:
gcloud app deploy --project MY_PROJECT -v test
you'll have both versions accessible at:
dev.myproject.appspot.com
test.myproject.appspot.com
Also, check Dan's answer below as it contains very relevant info.
To complement #MarCialR's answer, you can use SSL as well, but with *-dot-* URLs like https://test-dot-myproject.appspot.com, see Targeted routing.
But personally I'm not a big fan of a version-based environment, it brings trouble, see Continuous integration/deployment/delivery on Google App Engine, too risky?
I chose to implement environments at app level - each environment as a different app, each with its own URL. See Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?
Since I'm using a wildcard custom-domain SSL certificate I'm simply mapping the different apps (environments) to different hosts/subdomains in my custom domain - thus ensuring everything works as expected with a custom domain at every point in the CI/CD pipeline, no surprises in production.
I have a Flask-restful API(micro-service) written in python 2.7 using Google Cloud STANDARD environment, and dev_appserver.py(gcloud sdk) helps to run it locally.
Now, I plan to do three things:
Upgrade the code from Python 2.7 to Python 3.5
Google Cloud STANDARD environment to Google cloud FLEXIBLE environment
Question: but standard environment uses AppEngine library and methods, that don't work in flexible environment, so, can Google Cloud Client libraries solve the purpose?
Run locally and test the micro-service
Question: Please give suggestions to replace dev_appserver.py
There are a lot of differences between the environments. A lot of methods and libs from standard won't work in flex.
See here for more info:
https://cloud.google.com/appengine/docs/the-appengine-environments
https://cloud.google.com/appengine/docs/flexible/python/flexible-for-standard-users
https://cloud.google.com/appengine/docs/flexible/python/migrating
You can't use dev_appserver.py in flex environment. You will need to run Flask the regular way. e.g (locally) FLASK_APP=main.py FLASK_DEBUG=1 python -m flask run --port 8080
I've been struggling to find an online IDE which supports Go and Google App Engine. Codeenvy supports it using python, while Cloud9 runs python 2.6 (which isn't compatible with GAE), and the update doesn't work.
Is there any IDE that will allow me to develop and deploy Go to GAE?
Python 2.7.3 should be installable from command line in Cloud9 via c9pm install python27 (see also the github repo.
Koding comes with Go, Python2 & 3, and best of all gives you a normal Ubuntu VM with full root access! It will work great with Go and GAE :)