Google App Engine interactive console using the Terminal - google-app-engine

Following instructions on this page, Accessing the datastore remotely with remote_api:
I edited app.yaml to include these lines:
builtins:
- remote_api: on
I opened up a Terminal:
$ cd /path/to/app
$ python2.5 /usr/local/google_appengine/remote_api.shell.py\
-s localhost:8082 -p /_ah/remote_api
At the root of my app folder structure, I have a module named foobar.py, as well as a package named data_models. After gaining access to the remote_api Python interpreter, I try the following lines:
import foobar
import data_models
But I get an error:
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: No module named foobar
Traceback (most recent call last):
File "<console>", line 1, in <module>
ImportError: No module named data_models
How do I access the interactive console through the Terminal? Am I missing something? The same thing happens even if I login to the cloud server:
$ python2.5 /usr/local/google_appengine/remote_api.shell.py\
-s my-app.appspot.com -p /_ah/remote_api

remote_api just makes it possible to make RPC calls from a local task to a remote instance of an App Engine app. The Python console itself is still local, and everything you do executes locally. That means that any modules you try and import must exist on your local machine, somewhere your Python instance can find them - probably by adding your app's directory to PYTHONPATH, like this:
$ PYTHONPATH=/my/app/dir python2.5 /usr/local/google_appengine/remote_api.shell.py\
-s my-app.appspot.com -p /_ah/remote_api

I added the following lines to my .bash_profile so that I won't have to edit PYTHONPATH every time I fire up the Terminal.
PYTHONPATH = "/path/to/app"
export PYTHONPATH

It seems that the remote_api is in a different path than your app so it curpwd is that of the remote_api so your app's code is not available

Related

Using appengine with arch and zsh unable to find goapp

I am new to appengine and have installed google-cloud-sdk from the AUR(arch user repository) and and the google-appengine-go extention at /opt/google-cloud-sdk
thanks to this I am able to run a dev server using
dev_appserver.py app.yaml
But when using goapp serve I found
goapp: command not found
After adding /opt/google-cloud-sdk/platform/google_appengine:$PATH to my $PATH variable in zshrc and running goapp serve i now get the error.
zsh: permission denied: goapp
if sudo goapp serve
sudo: goapp: command not found
Due to this I am unable to use the updated sdk to run tests using goapp test
Thank you in advance for your help.
I had the same problem and I think I figured out how it usually works.
You download the google cloud sdk (https://cloud.google.com/sdk/downloads)
After downloading and unzipping to the folder where you want to use it you have to executet the ./google-cloud-sdk/install.sh.
Appengine is not part of the download.
It can be chosen with that install.sh script.
it will download items like appengine.
Afterwards you have a folder called
platform/google_appengine
as you mentioned yourself.
You might have to change execution permissions like
chmod 755 platform/google_appengine/go*
Add folder platform/google_appengine to the PATH if not done already.
The command "which" will not show non-executable binaries.
If you did not change permissions it will not show the path, even being within the PATH variable.

I'm new to command line. I get a lot of messages like 'command not found' and 'no such file or directory'

Trying to run gcloud init to initialize the Google App Engine Engine SDK by typing ./google-cloud-sdk/bin/gcloud init but it showed: no such file or directory or command not found. Is something wrong with my PATH? My path is:
/Users/AnneLutz/Documents/google-cloud-sdk\
If you typing ./google-cloud-sdk/bin/gcloud init and you installed Cloud SDK in /Users/AnneLutz/Documents/google-cloud-sdk, then your current directory should be /Users/AnneLutz/Documents in order for what you type to work.
That said you should add /Users/AnneLutz/Documents/google-cloud-sdk/bin to you path. To do this, assuming you are using bash you can
source /Users/AnneLutz/Documents/google-cloud-sdk/path.bash.inc
To make it so that every-time you start your shell you can add it to shell profile. For example you can add above source command at the end of ~/.bash_profile file.
It looks like you used the option to download the SDK zip file and are then trying to configure your environment with that download option. If you aren't comfortable with setting environment variables, you might want to instead try installing using the "interactive" installer, which will automate the steps for making the commands always available on your system.
The directions are here, but for Mac OS users are basically:
Enter the following at a command prompt:
curl https://sdk.cloud.google.com | bash
Restart your shell:
exec -l $SHELL
Run gcloud init to initialize the gcloud environment:
gcloud init
For many, this procedure is easier than getting everything configured manually.

Why is GOPATH not recognized by google-cloud-sdk go-app-stager?

When I try to update my appengine project, it gives me an error.
$ sudo gcloud app deploy
ERROR: (gcloud.app.deploy) Staging command [/usr/local/google-cloud-sdk/platform/google_appengine/goroot-1.6/bin/go-app-stager /Users/baz/app/app.yaml /tmp/tmpSZLK/tmpIQ4] failed with return code [1].
------------------------------------ STDOUT ------------------------------------
------------------------------------ STDERR ------------------------------------
2017/07/08 16:22:06 failed analyzing /Users/baz/app: cannot find package "bitbucket.org/foo/bar" in any of:
($GOROOT not set)
($GOPATH not set)
GOPATH:
$ echo $GOPATH
/Users/baz/go
$ go env GOPATH
/Users/baz/go
Clearly GOPATH is set (I export it in my .bash_profile).
TL;DR - Do not use sudo for deploying the app.
Longer version
You're running the gcloud app deploy command using sudo. sudo by the nature of the command will not retain the current set of environment variables when running the command as root user.
Try deploying your app engine app without sudo:
gcloud app deploy
If you do need to run you command as sudo (which is a smell that your setup is broken somehow), you can look at this post. Although, I would recommend looking into the possibility of running your command without sudo.

gcloud: how to download the app via cli

I depolyed an app with gcloud preview app deploy.
Is there a way to download it to an other local machine?
How can I get the files? I tried it via ssh with no success (can't access the docker dir)
UPDATE:
I found this:
gcloud preview app modules download default --version 1 --output-dir=my_dir
but it's not loading files
Log
Downloading module [default] to [my_dir/default]
Fetching file list from server...
|- Downloading [0] files... -|
I am coming to Google App Engine after two years, I see that they have made lots of improvements and added tons of features. But sadly, their documentation sometimes leaves much to be desired.
I used to download my code of the uploaded version with the appcfg.pyusing the following command.
appcfg.py download_app -A <app_id> -V <version> <output-dir>
But of course now that they have culminated everything in the gcloud shell where appcfg.py is not accessible.
However, the following method helped me to download the deployed code:
Go the console and in to the Google App Engine.
Select the project you want to work with.
Once the project's dashboard opens, Click on the top right to
open the built in console window.
Which should load the cloud shell at the bottom, now if you check appcfg.py is available to you to use in this VM.
Hence, use appcfg.py download_app -A <app_id> -V <version> <output-dir> to download the code.
Now once you have the code in the desired folder, in order to download it on your local machine - You can open the docker code editor
Now here I assumed if I rightclicked and exported the desired
folder it would work,
but instead it gave me the following error message.
{"Error":"'concurrency' must be a number but it is [object Undefined]","Message":"'concurrency' must be a number but it is [object Undefined]"}
So, I thought maybe it would play along nicely if the the folder
was an archive. Go back to the cloud shell and using whatever
utility you fancy make an archive of the folder
zip -r mycode.zip mycode
Go to the docker code editor, export and download.
Now. Of course there might many more ways do it (hopefully) but this is what made sense to me after returning to Google App Engine after 2 years.
Currently, the best way to do this is to pull the files out of Docker.
Put instance into self-managed mode, so that you can ssh into it:
$ gcloud preview app modules set-managed-by default --version 1 --self
Find the name of the instance:
$ gcloud compute instances list | grep gae-default-1
Copy it out of the Docker container, change the permissions, and copy it back to your local machine:
$ gcloud compute ssh --zone=us-central1-f gae-default-1-1234 'sudo docker cp gaeapp:/app /tmp'
$ gcloud compute ssh --zone=us-central1-f gae-default-1-1234 "chown -R $USER /tmp/app"
$ gcloud compute copy-files --zone=us-central1-f gae-default-1-1234:/tmp/app /tmp/
$ ls /tmp/app
Dockerfile
[...]
IMHO, the best option today (Aug 2018) is:
Under the main menu, under Products, go to Tools -> Cloud Build -> Build history.
There, click the ID of the build you want.
Then, in the opened window (Build details), click the source link, the download of your compressed code begins.
As simple as that.
HTH.
As of Feb 2021, you can install appengine-sdk using pip
pip install appengine-sdk
Once installed, appcfg can be used to download the app code.
python -m appcfg download_app -A app_id [ -V version ] out-dir
Nothing works. Finally I found the source code this way. Simply go to google cloud storage. choose buckets starting with us.artifacts...., select containers > images > download the latest one (look by created date). unzip after downloaded file. it will have all the deployed source code of app engine.

Jenkins gcloud deployment causes ImportError

I've set up a Jenkins instance in Google Compute Engine to build and deploy an App Engine Java project following this page from Google.
I've configured Jenkins to run the following shell command only when the Maven build succeeds: gcloud --project=decent-ellipse-843 preview app deploy target/*-SNAPSHOT/
When I attempt a build, the deployment fails with the following trace:
+ gcloud --project=decent-ellipse-843 preview app deploy target/backend-api-0.0.1-SNAPSHOT/
Traceback (most recent call last):
File "/usr/local/bin/../share/google/google-cloud-sdk/./lib/googlecloudsdk/gcloud/gcloud.py", line 183, in <module>
main()
File "/usr/local/bin/../share/google/google-cloud-sdk/./lib/googlecloudsdk/gcloud/gcloud.py", line 179, in main
_cli.Execute()
File "/usr/local/bin/../share/google/google-cloud-sdk/./lib/googlecloudsdk/calliope/cli.py", line 488, in Execute
post_run_hooks=self.__post_run_hooks)
File "/usr/local/bin/../share/google/google-cloud-sdk/./lib/googlecloudsdk/calliope/backend.py", line 1016, in Run
result = command_instance.Run(args)
File "/usr/local/bin/../share/google/google-cloud-sdk/./lib/googlecloudsdk/calliope/exceptions.py", line 86, in TryFunc
return func(*args, **kwargs)
File "/usr/local/share/google/google-cloud-sdk/lib/googlecloudsdk/appengine/app_commands/deploy.py", line 158, in Run
stage_dir = self.__MakeStagingDir(project, args, deployable)
File "/usr/local/share/google/google-cloud-sdk/lib/googlecloudsdk/appengine/app_commands/deploy.py", line 268, in __MakeStagingDir
java_app_update = appcfg_java.JavaAppUpdate(deployable, args)
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appcfg_java.py", line 146, in __init__
self.app_engine_web_xml = self._ReadAppEngineWebXml()
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appcfg_java.py", line 205, in _ReadAppEngineWebXml
parser=app_engine_web_xml_parser.AppEngineWebXmlParser)
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/appcfg_java.py", line 217, in _ReadAndParseXml
return parser().ProcessXml(file_handle.read())
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/app_engine_web_xml_parser.py", line 71, in ProcessXml
xml_root = ElementTree.fromstring(xml_str)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
parser = XMLParser(target=TreeBuilder())
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1466, in __init__
"No module named expat; use SimpleXMLTreeBuilder instead"
ImportError: No module named expat; use SimpleXMLTreeBuilder instead
Build step 'Execute shell' marked build as failure
Finished: FAILURE
The Maven build is successful, and I can deploy the project by issuing the gcloud command manually.
Even when I run the exact same gcloud command from the same directory and under the same user (tomcat), the deployment succeeds without errors.
I have re-installed python and updated the Google Cloud SDK without any results.
The instance is running Python 2.7.3, Jenkins 1.598, JDK 7u76 and Maven 3.2.2.
I hope someone can help me out with this!
I ran into a similar issue running my jobs on the cloud-dev-python slave. The images are missing build tools that you may need in order to build and deploy properly. My solution was to connect to the docker image and install the tools manually the first time around.
# on docker host, connect to java image
CONTAINER_ID=$(docker ps | grep cloud-dev-java | awk '{print $1}');
docker exec -i -t $CONTAINER_ID bash
# on docker image, install crap to build lxml, etc
gcloud -q components update preview app && apt-get update \
&& apt-get install -y build-essential libz-dev libxml2-dev \
libxslt1-dev python-dev python-pip && apt-get autoremove
Give that a whirl and see if it helps. I'm currently stuck with the service account not authenticating even though it is set up properly...

Resources