Google Cloud Endpoints files generated, server not working - google-app-engine

I run endpoints.sh
get-client-lib com.my.app.FooService and successfully generate the files for Google Cloud Endpoints[1] (2 .discovery files, 1 .api file and 1 .zip file).
The script doesn't add anything to my war folder, so I assume server-side is already handled by magic configured in web.xml and #Api annotations (a'la Spring Framework).
But http://localhost:8080/_ah/api/explorer redirects me to a blank Google cloud console. Uploading my app yields the same result.
Am I missing something? I think the documentation is a bit lacking, it doesn't even explain what the generated files are for.
I'm using Google App Engine Java.
[1] https://developers.google.com/appengine/docs/java/endpoints/gen_clients

To make the endpoints work, need follow the steps stated in the Google GAE page
Write your API backend code first.
Annotate your API backend code, so classes and client libraries can be generated from it. (Alternatively, use the Google Plugin for Eclipse, which annotates automatically for you.)
Generate the client library using the endpoints.sh utility. (Alternatively, use the Google Plugin for Eclipse to generate the client library.)
Write your client app, using the client library when making calls to the API backend.
But the above steps miss out some important steps, which the Google Plugins (Google Plugin for Eclipse) will generated for you automatically.
Configure the servlet in your web.xml. (Replace the your-full-class-name with your own class name)
<servlet>
<servlet-name>com.google.api.server.spi.SystemServiceServlet</servlet-name>
<servlet-class>com.google.api.server.spi.SystemServiceServlet</servlet-class>
<init-param>
<param-name>services</param-name>
<param-value>your-full-class-name</param-value>
</init-param>
</servlet>
<servlet-mapping>
<servlet-name>com.google.api.server.spi.SystemServiceServlet</servlet-name>
<url-pattern>/_ah/spi/*</url-pattern>
</servlet-mapping>
you need ensure the file (.api) generated by the endpoints.sh is copied into the folder WEB-INF in your web app root folder.
For better understanding, you can invoke the endpoints.sh to see all the available options as the following
Available commands:
get-client-lib: Generates a client library
usage: get-client-lib ...
Options:
--classpath=CLASSPATH Additional class path entries
-cp CLASSPATH (default: ./war/WEB-INF/classes).
--language=LANGUAGE The target output programming language
-l LANGUAGE (java) (default: java).
--output=OUTPUT_DIR The directory to store output files
-o OUTPUT_DIR (default: ./).
--war=WAR_PATH The path to a directory or .war with a WAR
-w WAR_PATH directory layout (default: ./war).
gen-api-config: Generates API configuration files from service classes
usage: gen-api-config ...
Options:
--classpath=CLASSPATH Additional class path entries
-cp CLASSPATH (default: ./war/WEB-INF/classes).
--output=OUTPUT_DIR The directory to store output files
-o OUTPUT_DIR (default: ./).
--war=WAR_PATH The path to a directory or .war with a WAR
-w WAR_PATH directory layout (default: ./war).
gen-discovery-doc: Generates API Discovery document
usage: gen-discovery-doc
Options:
--format=FORMAT The requested API protocol type (rest|rpc)
-f FORMAT (default: rest).
--output=OUTPUT_DIR The directory to store output files
-o OUTPUT_DIR (default: ./).
gen-client-lib: Generates a client library
usage: gen-client-lib
Options:
--language=LANGUAGE The target output programming language
-l LANGUAGE (java) (default: java).
--output=OUTPUT_DIR The directory to store output files
-o OUTPUT_DIR (default: ./).
Then you are able to view the services in the explorer UI as the following.
when you access the http: //localhost:8080/_ah/api/explorer, it will be redirected to the developers.google.com first. But the contents is from the localhost.
You can always rely on your normal Google APE local development server. For example, IntelliJ Google APE plugin start the Google APE local server also can support the "_ah/api/explorer" access

As mentioned by Dan Holevoet in the comment, the script needs to be ran from war. As for my case, I just need to copy the files into that directory.

Related

Unable to provide custom app.yaml file for Java GCP App Engine project

I have a Java 11 GCP App Engine project and I'm trying to use different app.yaml files depending on the environment (e.g., app-dev.yaml, app-prod.yaml, etc). The yaml files in the /appengine directory like src/main/appengine/app-dev.yaml and so on.
There is an SO post about this already, but the answer doesn't work because it clobbers the descriptor which in Java should be the pom.xml (see my Approach 2 for more information).
Approach #1
UPDATE: Solved! In order to use this approach you must be on gcloud 298.0.0+
First, I tried using the --appyaml=APPYAML argument found in https://cloud.google.com/sdk/gcloud/reference/app/deploy#--appyaml:
gcloud app deploy [DEPLOYABLES …] [--appyaml=APPYAML] [--bucket=BUCKET] ...
I ran the following and received an error that the appyaml argument isn't recognized.
$ gcloud --project=my-project app deploy --appyaml=app-dev.yaml
ERROR: (gcloud.app.deploy) unrecognized arguments: --appyaml=app-dev.yaml
The fully qualified path to app-dev.yaml doesn't work either.
Approach #2
Next I found a slighly different syntax in https://cloud.google.com/appengine/docs/flexible/java/configuring-your-app-with-app-yaml that looks like this:
gcloud app deploy service-name-app.yaml
I tried the same locally but pointed to my custom app-dev.yaml like so, but it breaks:
$ gcloud --project=my-project app deploy src/main/appengine/app-dev.yaml
...
descriptor: [/Users/SomeDev/IdeaProjects/my-project/app-server/src/main/appengine/app-dev.yaml]
source: [/Users/SomeDev/IdeaProjects/my-project/app-server/src/main/appengine]
target project: [my-project]
target service: [default]
target version: [20200831abcdefg]
target url: [https://my-project.uc.r.appspot.com]
This breaks because it thinks the app-dev.yaml is the descriptor file instead of a pom.xml, so it errors out with the following:
Error message: did not find any jar files with a Main-Class manifest entry
To compare, I ran a normal deployment without a custom yaml file and you can see the pom.xml is the value of the descriptor.
$ gcloud --project=my-project app deploy
...
descriptor: [/Users/SomeDev/IdeaProjects/my-project/app-server/pom.xml]
source: [/Users/SomeDev/IdeaProjects/my-project/app-server]
target project: [my-project]
target service: [default]
target version: [20200831abcdefg]
target url: [https://my-project.uc.r.appspot.com]
Is there a recommended way to make this work, or is this the wrong approach entirely?
Looking at your "Approach #1" you have to upgrade your gcloud to version >= 298.0.0 here --appyaml parameter have been added - quite recently in Jun'20.
Looking at your "Approach #2". If you are running gcloud app deploy (without parameters) it search for descriptor app.yaml in current directory and if not found - than for pom.xml. If you want to use pom.xml from different localization you have to remove it from current directory. I didn't test it to the very end, just tested the descriptor value in summary.
Anyway I don't think using above is best way to do it. When you use pom.xml as descriptor it means that you are using feature called "deploy your Maven project as source code". Which is not main way to deploy app engine with maven.
According to my understanding if maven was used for build, its possible to use the jar in entrypoint of app.yaml file (reference) or maven goal appengine:deploy (reference + article that should be interesting).

Spacy on AppEngine standard

I'm trying to use Spacy on the new AppEngine Standard Python 3.7 runtime.
When I try to deploy I get:
ERROR: (gcloud.app.deploy) Cannot upload file
[/my/project/path/venv/lib/python3.7/site-packages/spacy/lang/tr/lemmatizer.py],
which has size [41523943] (greater than maximum allowed size of
[33554432]). Please delete the file or add to the skip_files entry in
your application .yaml file and try again.
A few oddities:
The docs seem to indicate that I don't need to upload the virtual environment and it will be created from requirements.txt
Looking at the log file, it seems to ignore .pyc files, but not the venv directory
The error message says to add to the skip_files in your application .yaml file and try again., but the docs say the python3.7 runtime doesn't use skip files and to use a .gcloudignore file instead, but adding venv/ or venv/* doesn't work (it appears to be ignored)
To fix this, I needed up update gcloud and reauthenticate:
gcloud components update
gcloud auth login

Jdeveloper war deployment to tomcat

I am a newbie.
I am trying to publish procedures which return refcursor as webservice.
As webservices do not support ref cursor return type so I chnaged the return type to pl/sql table.
I created the jax rpc in Jdeveloper studio 11.1.2.4.0 and tested it. This runs ok. I need to deploy it on tomcat server and am failing completely.
I copied the war file generated by Jdeveloper to the webapps directory in Tomcat and started Tomcat but this didn't work, as the following exception was thrown:
java.lang.ClassCastException: omsdevconn.MyWebServicetomcatUser cannot be cast to javax.servlet.Servlet
If someone has done it then can I get the steps for the process to deploy it on Tomcat 6.0.37?
Thanks in advance
Download jaxws-rt-2.0-SNAPSHOT.jar and jaxws-2_0.jar
Copy into $CATALINA_HOME/lib folder
Add the following in web.xml under web-app
<web-app>
**<listener>
<listener-class>
com.sun.xml.ws.transport.http.servlet.WSServletContextListener
</listener-class>
</listener>**
............<!--your other servlet definations-->
</web-app>
create sun-jaxws.xml under your applications web-inf folder and configure endpoints.You can have a look at jdeveloper generated xml file to get all the information
sample sun-jaxws.xml
<?xml version="1.0" encoding="UTF-8"?>
<endpoints xmlns="http://java.sun.com/xml/ns/jax-ws/ri/runtime"version="2.0">
<endpoint name="fromwsdl"
interface="com.foo.yourserviceInterface"
implementation="com.foo.yourserviceClass"
wsdl="WEB-INF/wsdl/youservice.wsdl"
service="YourService"
port="YourServicePort"
url-pattern="/serviceAccessUrl" />
</endpoints>
create war for your application and deploy you application under tomcat/webapps
start tomcat
After starting tomcat you can access the service as
http:/host:port/yourservicename/methodInService
As you are calling oracle plsql code and IF you are using any oracle specific calls like OracleCallabelStatement then on run time you will get class cast exceptions from tomcat datasource when trying to cast callabelstatement to oraclecallabelstatement for that you need to configure oracledatasource in tomcat
http://www.oracle.com/technetwork/articles/oem/ucp-jdbc-tomcat-355431.html
More information on Java API for XML Web Services (JAX-WS)
https://jax-ws.java.net/nonav/2.2.1/docs/UsersGuide.html#1.0_Introduction
Another way to deploy service generated by Jdeveloper when exposing a plsql procedure as webservice
I am not allowed to post more that 2 links so please add and extra "/" after http: to correct the links
This solution uses Apache Axis which allows you to call your web service(which is not exposed as restful) directly from browser by creating a SOAP request behind the scenes and involke the service method giving it soap message.
1)Generate a webservice in Jdeveloper .It will create java files and corresponding ".class" file , "wsdl file" in web-inf and some other xml files. Make a note of LOCATION of .class file and wsdl file.
2)Download eclipse and extract it on your hard drive (It needs java to run So you must have java installed on system .You can download eclipse from
http:/www.eclipse.org/downloads/packages/eclipse-ide-java-ee-developers/junosr1
3)Download Axis plugin for eclipse from here :
http:/www.apache.org/dyn/mirrors/mirrors.cgi/axis/axis2/java/core/1.6.2/axis2-eclipse-service-plugin-1.6.2.zip
and install it in eclipse. Installation Instructions :
http:/axis.apache.org/axis2/java/core/tools/eclipse/plugin-installation.html
4)Generate the aar file via eclipse axis plugin. Instructions :
http:/axis.apache.org/axis2/java/core/tools/eclipse/servicearchiver-plugin.html#Operation
5)Make following changes in tomcat-user.xml file
<role rolename="tomcat"/>
<role rolename="manager-gui"/>
<role rolename="admin-gui"/>
<role rolename="manager-jmx"/>
<role rolename="role1"/>
<user username="tomcat" password="tomcat" roles="tomcat,manager-gui,admin-gui,manager-jmx"/>
6)IF YOU ARE USING ORACLE SPECIFIC CLASSES IN CODE (Which most probably Jdeveloper will produce) : Configure a oracle data source in tomcat (server.xml or context.xml). Instructions :
http:/www.oracle.com/technetwork/articles/oem/ucp-jdbc-tomcat-355431.html
Make sure you have ojdbc6.jar and ucp.jar in Tomcat/lib folder.
7)Download axis war file to deploy in tomcat from here :
http:/www.apache.org/dyn/mirrors/mirrors.cgi/axis/axis2/java/core/1.6.2/axis2-eclipse-service-plugin-1.6.2.zip
Unzip the zip , it will have a axis2.war file in it. Put this war file under “\Tomcat\webapps” directory
8)Start tomcat. On tomcat console see if there are any errors in deploying axis. If everything is fine you should be able to see axis page on http:/yourhost:yourport/axis2/
9)Click on Administration link . Enter user id “admin” and password “Axis2” and login
10)Click upload and browse to the location of aar file you generate in step 4 and upload. After it shows you success click on Available service. It takes a bit of time but you should be able to see you service.Click on service to see wsdl to verify
11)Call your service method from browser and cross your fingers and toes.
Note that the link to hit will be something Like
http:/yourhost:yourport/axis2/services/YourServiceName/serviceMethod?arg1=blah&
Make sure your got the arg names right.you should be able to see the correct names from wsdl

Google App Engine remote_api script not found

I'm setting up remote_api locally and this time around it's not working. I'm just following the instructions on the remote_api doc page for python here: http://code.google.com/appengine/articles/remote_api.html
Which basically means I'm running the following command from the project app root (that contains app.yaml)
>> python $GAE_SDK_ROOT/remote_api_shell.py -s your_app_id.appspot.com
>> /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python: can't open file '/remote_api_shell.py': [Errno 2] No such file or directory
The your_app_id param is being changed to the actual app id on google's.
It's probably a simple thing, but not sure what it is.
You need to replace $GAE_SDK_ROOT with the actual root directory of your GAE SDK. So, probably:
python ~/google_appengine/remote_api_shell.py -s ...

How do I download the source code of a google app engine project?

This seems like it should be very easy but I don't see a link to it anywhere.
How do I download the source code of a google app engine project?
Windows
appengine-java-sdk\bin\appcfg.cmd -A <your_app_id> -V <your_app_version> download_app <output-dir>
Linux
./appengine-java-sdk/bin/appcfg.sh -A <your_app_id> -V <your_app_version> download_app <output-dir>
For completeness, using the Python implementation:
appcfg.py download_app -A $appID -V $appVersionNumber $downloadDirectory --oauth2
--oauth2 is of course optional, you can omit it and provide your email + app-specific password (or your password, and then go implement two-factor authentication right after), but it's easier, and frankly there's no reason not to.
Documentation.
App Engine actually recently added the ability for the developer who uploaded a given app version to download its source code.
As of October 2019 you can simply go to --> App Engine --> Services and in the tool dropdown select 'source' and the source code is there
Posting this since none of the listed methods above didn't take me to the code (by June 2021)
You could try accessing it through;
Google Cloud Platform > Debugger > choosing the version of the
Application from combo at top.
This will list the files of that version on the left pane. There is no way to download it automatically but you can copy-paste the code.
Hope you will find this helpful.
IMHO, the best option today (Aug 2018) is:
Under the main menu, under Products, go to Tools -> Cloud Build -> Build history.
There, click the ID of the build you want (for me - the last one).
Then, in the opened window (Build details), click the "source" link, the download of your compressed code begins.
As simple as that.
HTH.
Working with App engine standard using Go, the debugger isn't available yet.
How I managed to download the source code for an existing service was to use the gcloud tool.
First: Get the version id of your service using the app engine console or running: gcloud app versions list
Second: use the version and service name and run: gcloud app versions describe <versionID> --service=<service name>
the describe parameter will give you the storage locations for your source files that looks like this:
cmd/main.go:
sha1Sum: e3fe5848c2640eca7ac3591490e1debc2d3a9b09
sourceUrl: https://storage.googleapis.com/<project>/<file id>
Third: you can then use the storage console, using the file id, to download the files you are interested in.
this process based on java sdk
Its works for me...
Download Google cloud SDK
gcloud init
enter image description here
Follow through process of logging in using your credentials
Enter following command from SDK
C:\Program Files (x86)\Google\appengine-java-sdk-1.9.49\bin
enter image description here
Enter Following command to download source code
appcfg.sh -A [YOUR_APP_ID] -V [YOUR_APP_VERSION] download_app [OUTPUT_DIR]
Eg: appcfg.sh -A my-project-name-1234 -V 2 download_app C:\Users\india\Desktop\my project
Note: this progress based on java-appengine sdk so we use appcfg.sh instead of appcfg.py
check if your app is uploaded with same email id that is in your app engine. if you are not sure then in app engine > control > Clear deployment credentials and then click on any project, deploy to sign in again then use this
appcfg.py download_app -A {app id from google app engine} -V {1} "{c:\path}" --oauth2_credential_file=C:\Users\{your account name}/.appcfg_oauth2_tokens
change all {} to your needs
Things have changed since this question was asked so I'm adding an updated answer. Note that this only applies to GAE Standard Environment
Google has deprecated appcfg.py and so the previous responses appcfg.py download_app no longer works.
gcloud which is the SDK in use (it replaced appcfg) does not have the functionality to download your source code.
When you deploy your app via gcloud app deploy, it copies your source code to a bucket. The default bucket is staging.<project_name>.appspot.com. Your files will stay in this bucket for a maximum of 15 days before they are deleted. You can modify the rule so that the files are retained for longer or less time.
The file names in the bucket are encoded so you can't figure out what each file is unless you open it (i.e. download it). Google has a mapping of the encoded names to the original file names. To get this mapping, you run the gcloud app versions describe command and it will list the file names and their encoded names. To download the files, you have to manually click each url one by one. So essentially, you have to download each file manually and then use the mapping to rename them (or open the file, check the content and then rename them). Also note that downloading the files manually will not maintain the folder structure in which they were uploaded.
If you do not wish to go through all of the above hassles (imagine having to manually open each url for each file if you have a small to mid-sized project which has hundreds of files), our App - https://nocommandline.com - now supports downloading source code from the default bucket - staging.<project_name>.appspot.com (so far as your files are still there which means any deployment i.e update not older than 15 days from your current date unless you previously increased the deletion age on your staging bucket's lifecycle page).
In simple terms, you enter your project name, the version number and our App will take care of retrieving the original file name to encoded name mapping, automatically downloading the files and renaming them to the original names, while maintaining the folder structure. For more information, refer to https://nocommandline.com/help/#faq_download_source_code_from_gae.
Log in to the console.developers.google.com
Select the project you want to download the code from (Google App Engine Standard Envoronment).
Go to the App Engine Dashboard. Under Summary is Debug and Source. Click on Source.
Select each file one at a time and copy it (highlight the code, copy and paste into your local editor.)
Select the next file....
You need to use svn to checkout the files.
If you are on Windows, you can use tortoise svn for your GUI end.
Here are tutorials on how to do it, here is the related question.

Resources