deploy scala sbt to google app engine - google-app-engine

While importing my scala sbt project to intellij i get the following error
Error while importing SBT project: ...
[info] Loading project definition from /home/dnilesh/Desktop/app-engine/sbt-appengine-demo-master/project
java.lang.RuntimeException: You need to set APPENGINE_SDK_HOME
at scala.sys.package$.error(package.scala:27)
at sbtappengine.Plugin$AppEngine$.buildAppengineSdkPath(AppenginePlugin.scala:103)
at sbtappengine.Plugin$$anonfun$baseAppengineSettings$33.apply(AppenginePlugin.scala:219)
at sbtappengine.Plugin$$anonfun$baseAppengineSettings$33.apply(AppenginePlugin.scala:219)
at sbt.Init$Value$$anonfun$apply$13.apply(Settings.scala:604)
at sbt.EvaluateSettings$$anonfun$sbt$EvaluateSettings$$constant$1.apply(INode.scala:163)
at sbt.EvaluateSettings$$anonfun$sbt$EvaluateSettings$$constant$1.apply(INode.scala:163)
at sbt.EvaluateSettings$MixedNode.evaluate0(INode.scala:175)
at sbt.EvaluateSettings$INode.evaluate(INode.scala:135)
at sbt.EvaluateSettings$$anonfun$sbt$EvaluateSettings$$submitEvaluate$1.apply$mcV$sp(INode.scala:69)
at sbt.EvaluateSettings.sbt$EvaluateSettings$$run0(INode.scala:78)
at sbt.EvaluateSettings$$anon$3.run(INode.scala:74)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[error] You need to set APPENGINE_SDK_HOME
[error] Use 'last' for the full log.
Invalid response.
See complete log in file:/home/dnilesh/.IdeaIC2017.2/system/log/sbt.last.log Consult IDE log for more details (Help | Show Log)
and while deploying same app to google app engine from terminal i get the same error after that, In terminal i set APPENGINE_SDK_HOME to my google cloud installation directory /usr/lib/google-cloud-sdk/platform/google_appengine/. then i get the following error.
So how can i set APPENGINE_SDK_HOME and to which path so that sbt appengineDeploy command will work.
root#dnilesh-pc:/home/dnilesh/Desktop/app-engine/sbt-appengine-demo-master# sbt appengineDeploy
[info] Loading project definition from /home/dnilesh/Desktop/app-engine/sbt-appengine-demo-master/project
java.lang.RuntimeException: not found appengine api jar.
at scala.sys.package$.error(package.scala:27)
at sbtappengine.Plugin$AppEngine$.buildSdkVersion(AppenginePlugin.scala:114)
at sbtappengine.Plugin$$anonfun$baseAppengineSettings$32.apply(AppenginePlugin.scala:218)
at sbtappengine.Plugin$$anonfun$baseAppengineSettings$32.apply(AppenginePlugin.scala:218)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.EvaluateSettings$MixedNode.evaluate0(INode.scala:175)
at sbt.EvaluateSettings$INode.evaluate(INode.scala:135)
at sbt.EvaluateSettings$$anonfun$sbt$EvaluateSettings$$submitEvaluate$1.apply$mcV$sp(INode.scala:69)
at sbt.EvaluateSettings.sbt$EvaluateSettings$$run0(INode.scala:78)
at sbt.EvaluateSettings$$anon$3.run(INode.scala:74)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[error] not found appengine api jar.
[error] Use 'last' for the full log.

Steps I did to sort out this issue:
Download the App Engine SDK for Java:
Google App Engine
Direct Download Link
Extract it in your computer regardless your GCloud installed path.
set APPENGINE_SDK_HOME :
export APPENGINE_SDK_HOME= /PATH/TO/SDK/appengine-java-sdk-1.9.54
Download Sample Scala sbt application.
Github Sample Scala App
run sbt appengineDeploy command to run app. change project id to your project id on google cloud. Steps before running app
Notes:
You cannot install third party databases like mongodb on gae.
gae does not support akka framework.
gae does not support multithreading.

Related

Unable to provide custom app.yaml file for Java GCP App Engine project

I have a Java 11 GCP App Engine project and I'm trying to use different app.yaml files depending on the environment (e.g., app-dev.yaml, app-prod.yaml, etc). The yaml files in the /appengine directory like src/main/appengine/app-dev.yaml and so on.
There is an SO post about this already, but the answer doesn't work because it clobbers the descriptor which in Java should be the pom.xml (see my Approach 2 for more information).
Approach #1
UPDATE: Solved! In order to use this approach you must be on gcloud 298.0.0+
First, I tried using the --appyaml=APPYAML argument found in https://cloud.google.com/sdk/gcloud/reference/app/deploy#--appyaml:
gcloud app deploy [DEPLOYABLES …] [--appyaml=APPYAML] [--bucket=BUCKET] ...
I ran the following and received an error that the appyaml argument isn't recognized.
$ gcloud --project=my-project app deploy --appyaml=app-dev.yaml
ERROR: (gcloud.app.deploy) unrecognized arguments: --appyaml=app-dev.yaml
The fully qualified path to app-dev.yaml doesn't work either.
Approach #2
Next I found a slighly different syntax in https://cloud.google.com/appengine/docs/flexible/java/configuring-your-app-with-app-yaml that looks like this:
gcloud app deploy service-name-app.yaml
I tried the same locally but pointed to my custom app-dev.yaml like so, but it breaks:
$ gcloud --project=my-project app deploy src/main/appengine/app-dev.yaml
...
descriptor: [/Users/SomeDev/IdeaProjects/my-project/app-server/src/main/appengine/app-dev.yaml]
source: [/Users/SomeDev/IdeaProjects/my-project/app-server/src/main/appengine]
target project: [my-project]
target service: [default]
target version: [20200831abcdefg]
target url: [https://my-project.uc.r.appspot.com]
This breaks because it thinks the app-dev.yaml is the descriptor file instead of a pom.xml, so it errors out with the following:
Error message: did not find any jar files with a Main-Class manifest entry
To compare, I ran a normal deployment without a custom yaml file and you can see the pom.xml is the value of the descriptor.
$ gcloud --project=my-project app deploy
...
descriptor: [/Users/SomeDev/IdeaProjects/my-project/app-server/pom.xml]
source: [/Users/SomeDev/IdeaProjects/my-project/app-server]
target project: [my-project]
target service: [default]
target version: [20200831abcdefg]
target url: [https://my-project.uc.r.appspot.com]
Is there a recommended way to make this work, or is this the wrong approach entirely?
Looking at your "Approach #1" you have to upgrade your gcloud to version >= 298.0.0 here --appyaml parameter have been added - quite recently in Jun'20.
Looking at your "Approach #2". If you are running gcloud app deploy (without parameters) it search for descriptor app.yaml in current directory and if not found - than for pom.xml. If you want to use pom.xml from different localization you have to remove it from current directory. I didn't test it to the very end, just tested the descriptor value in summary.
Anyway I don't think using above is best way to do it. When you use pom.xml as descriptor it means that you are using feature called "deploy your Maven project as source code". Which is not main way to deploy app engine with maven.
According to my understanding if maven was used for build, its possible to use the jar in entrypoint of app.yaml file (reference) or maven goal appengine:deploy (reference + article that should be interesting).

How call GAE EndpointV2 from Java Client

How can I call GAE Endpoints V2 from my Java Client using google-api-client:1.23.0 library.
I followed the migration description (https://cloud.google.com/endpoints/docs/frameworks/java/migrating) for the backend . Deployment worked fine.
But using the generated endpoints client libs I get the error:
Exception in thread "LWJGL Application" com.badlogic.gdx.utils.GdxRuntimeException: java.lang.NoSuchMethodError: com.google.api.client.googleapis.services.json.AbstractGoogleJsonClient$Builder.setBatchPath(Ljava/lang/String;)Lcom/google/api/client/googleapis/services/AbstractGoogleClient$Builder;
at com.badlogic.gdx.backends.lwjgl.LwjglApplication$1.run(LwjglApplication.java:133)
Caused by: java.lang.NoSuchMethodError: com.google.api.client.googleapis.services.json.AbstractGoogleJsonClient$Builder.setBatchPath(Ljava/lang/String;)Lcom/google/api/client/googleapis/services/AbstractGoogleClient$Builder;
at com.example.qwarks.backend.qwarksApi.QwarksApi$Builder.setBatchPath(QwarksApi.java:2098)
at com.example.qwarks.backend.qwarksApi.QwarksApi$Builder.<init>(QwarksApi.java:2077)
at com.example.qwarks.utility.Api.<clinit>(Api.java:24)
at com.example.qwarks.screens.SplashScreen.<init>(SplashScreen.java:105)
at com.example.qwarks.Qwarks.create(Qwarks.java:59)
at com.badlogic.gdx.backends.lwjgl.LwjglApplication.mainLoop(LwjglApplication.java:149)
at com.badlogic.gdx.backends.lwjgl.LwjglApplication$1.run(LwjglApplication.java:126)
I found the problem, I had to remove
compile project(path: ':backend', configuration: 'endpoints')
from my gradle dependencies

Eclipse Oxygen and Google Cloud Tools Missing javax.servlet Classes from Build Path

I have a GAE project in Eclipse that has been working without a problem. I recently upgraded to the Google Cloud SDK 179.0.0 and Eclipse Oxygen.1a Release (4.7.1a) and now, when I open the project, I'm getting build path errors on classes in the javax.servlet.http package. Specifically, the following classes are missing from my project's build path.
javax.servlet.ServletContext
javax.servlet.http.HttpServlet
If you start Eclipse from the console using the following commands (assuming Eclipse is installed in your /Applications folder):
cd /Applications/Eclipse.app/Contents/MacOS
./eclipse
And you watch the terminal window, you'll see a bunch of stack traces similar to the following:
Nov 10, 2017 9:24:57 AM com.google.cloud.tools.eclipse.appengine.localserver.ServletClasspathProvider doResolveClasspathContainer
WARNING: Failed to initialize libraries
org.eclipse.core.runtime.CoreException: Could not resolve artifact com.google.appengine:appengine-api-1.0-sdk:jar:1.9.57
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:776)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$5.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:743)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.resolve(MavenImpl.java:720)
at com.google.cloud.tools.eclipse.util.MavenUtils.resolveArtifact(MavenUtils.java:69)
at com.google.cloud.tools.eclipse.appengine.libraries.repository.MavenHelper.resolveArtifact(MavenHelper.java:46)
at com.google.cloud.tools.eclipse.appengine.libraries.repository.M2RepositoryService.resolveArtifact(M2RepositoryService.java:50)
at com.google.cloud.tools.eclipse.appengine.libraries.repository.LibraryClasspathContainerResolverService.resolveLibraryFileAttachSourceSync(LibraryClasspathContainerResolverService.java:245)
at com.google.cloud.tools.eclipse.appengine.libraries.repository.LibraryClasspathContainerResolverService.resolveLibraryAttachSourcesSync(LibraryClasspathContainerResolverService.java:106)
at com.google.cloud.tools.eclipse.appengine.localserver.ServletClasspathProvider.doResolveClasspathContainer(ServletClasspathProvider.java:100)
at com.google.cloud.tools.eclipse.appengine.localserver.ServletClasspathProvider.resolveClasspathContainer(ServletClasspathProvider.java:79)
at org.eclipse.jst.server.core.RuntimeClasspathProviderDelegate.resolveClasspathContainerImpl(RuntimeClasspathProviderDelegate.java:135)
at org.eclipse.jst.server.core.internal.RuntimeClasspathProviderWrapper.resolveClasspathContainerImpl(RuntimeClasspathProviderWrapper.java:155)
at org.eclipse.jst.server.core.internal.RuntimeClasspathContainer.getClasspathEntries(RuntimeClasspathContainer.java:77)
at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:2693)
at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:2851)
at org.eclipse.jdt.internal.core.JavaProject.getResolvedClasspath(JavaProject.java:1956)
at org.eclipse.jdt.internal.core.ExternalFolderChange.updateExternalFoldersIfNecessary(ExternalFolderChange.java:39)
at org.eclipse.jdt.internal.core.ChangeClasspathOperation.classpathChanged(ChangeClasspathOperation.java:59)
at org.eclipse.jdt.internal.core.SetContainerOperation.executeOperation(SetContainerOperation.java:111)
at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:724)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2240)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2267)
at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:795)
at org.eclipse.jdt.internal.core.JavaModelManager$10.run(JavaModelManager.java:3001)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2240)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:2267)
at org.eclipse.jdt.internal.core.JavaModelManager.initializeAllContainers(JavaModelManager.java:3017)
at org.eclipse.jdt.internal.core.JavaModelManager.getClasspathContainer(JavaModelManager.java:2038)
at org.eclipse.jdt.core.JavaCore.getClasspathContainer(JavaCore.java:3497)
at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:2687)
at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:2851)
at org.eclipse.jdt.internal.core.JavaProject.getResolvedClasspath(JavaProject.java:1956)
at org.eclipse.jdt.internal.core.DynamicProjectReferences.getDependentProjects(DynamicProjectReferences.java:34)
at org.eclipse.core.internal.resources.ProjectDescription.computeDynamicReferencesForProject(ProjectDescription.java:950)
at org.eclipse.core.internal.resources.ProjectDescription.getAllBuildConfigReferences(ProjectDescription.java:265)
at org.eclipse.core.internal.resources.ProjectDescription.getAllReferences(ProjectDescription.java:221)
at org.eclipse.core.internal.resources.Project.getReferencedProjects(Project.java:469)
at org.eclipse.ui.actions.CloseUnrelatedProjectsAction.buildConnectedComponents(CloseUnrelatedProjectsAction.java:78)
at org.eclipse.ui.actions.CloseUnrelatedProjectsAction.computeRelated(CloseUnrelatedProjectsAction.java:195)
at org.eclipse.ui.actions.CloseUnrelatedProjectsAction.getSelectedResources(CloseUnrelatedProjectsAction.java:211)
at org.eclipse.ui.actions.SelectionListenerAction.selectionIsOfType(SelectionListenerAction.java:240)
at org.eclipse.ui.actions.CloseResourceAction.updateSelection(CloseResourceAction.java:187)
at org.eclipse.ui.actions.BaseSelectionListenerAction.selectionChanged(BaseSelectionListenerAction.java:124)
at org.eclipse.ui.internal.navigator.resources.actions.ResourceMgmtActionProvider.updateActionBars(ResourceMgmtActionProvider.java:254)
at org.eclipse.ui.internal.navigator.resources.actions.ResourceMgmtActionProvider.fillActionBars(ResourceMgmtActionProvider.java:84)
at org.eclipse.ui.navigator.NavigatorActionService$3.run(NavigatorActionService.java:265)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at org.eclipse.ui.navigator.NavigatorActionService.fillActionBars(NavigatorActionService.java:258)
at org.eclipse.ui.navigator.CommonNavigatorManager$UpdateActionBarsJob$1.run(CommonNavigatorManager.java:95)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at org.eclipse.ui.navigator.CommonNavigatorManager$UpdateActionBarsJob.runInUIThread(CommonNavigatorManager.java:89)
at org.eclipse.ui.progress.UIJob.lambda$0(UIJob.java:95)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:37)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:182)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4033)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3700)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1155)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1044)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:153)
at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:680)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:594)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:151)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:653)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:590)
at org.eclipse.equinox.launcher.Main.run(Main.java:1499)
Contains: Missing com.google.appengine:appengine-api-1.0-sdk:jar:1.9.57
With the 179.0 version of the Google Cloud SDK installed, I now have App Engine 1.9.59 installed versus 1.9.57.
The Eclipse build path contains the following libraries:
App Engine Standard Runtime [App Engine Standard Runtime]
EAR Libraries
JRE System Library [JavaSE-1.8]
TestNG
Web App Libraries
I encounter this same problem with the following Eclipse releases:
4.7.0 (Build Id: 20170620-1800)
4.7.1a (Build Id: 20171005-1200)
The following is a list of errors my project now has:
RequestDispatcher cannot be resolved to a type
The project was not built since its build path is incomplete. Cannot find the class for javax.servlet.ServletContext. Fix the build path then try building this project
The superclass "javax.servlet.http.HttpServlet" was not found on the Java Build Path
The type javax.servlet.ServletContext cannot be resolved. It is indirectly referenced from required .class files
Creating a new App Engine Standard Edition project as a Maven project specifies a dependency on the 1.9.57 version of the App Engine API.
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-api-1.0-sdk</artifactId>
<version>1.9.57</version>
<scope>provided</scope>
</dependency>
Created issue #2754 for this problem on GitHub under the "GoogleCloudPlatform/google-cloud-eclipse" project.
https://github.com/GoogleCloudPlatform/google-cloud-eclipse/issues/2574
Version Information
Eclipse Oxygen.1a Release (4.7.1a Build: 20171005-1200)
Google Cloud Platform for Eclipse 4.5 and later 1.4.1.201710261936
Google Cloud SDK 179.0.0
alpha 2017.09.15
app-engine-java 1.9.59
app-engine-python 1.9.62
beta 2017.09.15
bq 2.0.27
core 2017.11.06
gcd-emulator v1beta3-1.0.0
gcloud
gsutil 4.28
kubectl
pubsub-emulator 2017.09.15
I don't know the exact cause of the problem, but rebooting my computer resolved the problem reported in this bug. In the course of trying to resolve this problem, I did stop and start Eclipse several times in an effort to address the problem. I started new terminal window sessions in an effort to fix the problem, but that didn't work.
After completely rebooting my machine, the problem reported in this bug went away.

Apache Flink Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Traversable

I have just started learning Apache Flink and found the guide link to start the development in EClipse IDE.
I followed the this to start off but getting the below error
00:20:26,993 INFO org.apache.flink.api.java.ExecutionEnvironment - The job has 0 registered types and 0 default Kryo serializers
Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Traversable
at java.lang.ClassLoader.defineClass1(Native Method)
Here I have place the Error log log File ... Please let me know if your require more details. Thanks, Nyamath
java.util.zip.ZipException: invalid LOC header (bad signature)
Your Scala jar file provided by Maven seems to be corrupted. Please update your Maven dependencies by executing this from your project folder on the command line:
mvn -U clean install
In Eclipse, right click on your project and click on Update - - > Maven dependencies.
If that does not work you'll need to delete the corrupted Jar file in the .m2/repositories/ folder.

Trouble with Google appengine id

I successfully launched an app locally on the local host
When I deployed to the appengine I got an error.
com.google.appengine.tools.admin.HttpIoException: Error posting to URL: https://appengine.google.com/api/appversion/create?app_id=my_id&version=1&
404 Not Found
This application does not exist (app_id=u'my_id').
For some reason the ID given in the error has u and a space appended in the beginning
I doubled checked the application exists and succesfuly uploaded the app from another computer
All the solutions I can find (like "This application does not exist (app_id=xxx)", or "GAE - Deployment Error: “AttributeError: can't set attribute”") mention running appcfg.py with the --no_cookies
C:\Program Files (x86)\Google\google_appengine\appcfg.py
appcfg.py --no_cookies update my-app-folder\
# or (2-steps verificatio enabled)
appcfg.py --oauth2 --no_cookies update my-app-folder\
If the project contains Java7 classes, you need to add the --use_java7 option, as shown in the question "Failed to deploy to Google App Engine because --use_java7 flag has not been set".
While you can run appcfg from the GAE installation path, the page "Using the Google Plugin for Eclipse" mentions (at the bottom of the page):
Some features of the App Engine Java SDK tools are only available by running the tools directly from the command line. If you have installed the SDK using Eclipse, you can run these tools from the Eclipse plugin installation directory.
The SDK is located in your Eclipse installation directory, under plugins/com.google.appengine.eclipse.sdkbundle_VERSION/, where VERSION is a version identifier for the SDK.
In this directory is the appengine-java-sdk/bin/ subdirectory containing the tools.

Resources