How to access memorystore from App engine standard environment? - google-app-engine

We deploy our app on Google app engine standard environment. We need to access Memorystore(redis) from our app engine.
Following the document, we create Serverless VPC access connector and configure the app engine:
<vpc-access-connector>
<name>projects/PROJECT_ID/locations/REGION/connectors/CONNECTOR_NAME</name>
</vpc-access-connector>
and set the IAM permissions. But we still can not connect to redis instance at private IP like 10.0.0.4 using jedis:
Jedis jedis = new Jedis("10.0.0.4");

It should work if you deploy it with gcloud beta app deploy target/SNAPSHOT.
I prepared and uploaded a sample in Github.
How I did it in a new project:
Enabled App Engine, selected region us-central (corresponds to us-central1)
Created Memorystore instance, in region us-central1
Created VPC Connector, in region us-central1 (At the moment no other region can be selected, so both App Engine and Memorystore have to be in us-central1)
Added the VPC connector in appengine-web.xml:
<vpc-access-connector>
<name>projects/PROJECT_ID/locations/us-central1/connectors/CONNECTOR_NAME</name>
</vpc-access-connector>
Modified pom.xml, adding the following dependency:
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>3.1.0</version>
<type>jar</type>
<scope>compile</scope>
</dependency>
Modified the servlet.java file:
import java.io.IOException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import redis.clients.jedis.Jedis;
#WebServlet(
name = "Redis",
description = "Redis: Connect to Redis",
urlPatterns = "/redis"
)
public class RedisServlet extends HttpServlet {
#Override
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
String s;
try{
Jedis jedis = new Jedis("10.0.0.4");
jedis.set("myKey", "It's alive");
s = "Life test: "+ jedis.get("myKey");
}catch(Exception e){s = "Couldn't connect "; e.printStackTrace();}
resp.getWriter().write(s);
}
}
Ran the following to package and deploy:
mvn package (This will create a "target" folder)
gcloud beta app deploy target/ARTIFACT_ID-1.0-SNAPSHOT
Note that it's still in BETA and it might not work very reliably.

Related

Spring Boot App Engine log messages not showing

I have a Spring Boot application that I am deploying to an App Engine standard environment using Java 8. I cannot seem to get log messages to show in the log viewer in my cloud console. I do have other logs working such as the endpoint being hit.
logging.properties:
.level = FINEST
appengine-web.xml:
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<threadsafe>true</threadsafe>
<runtime>java8</runtime>
<service>logging-service</service>
<system-properties>
<property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
</system-properties>
</appengine-web-app>
Spring Controller:
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.util.logging.Level;
import java.util.logging.Logger;
import static org.springframework.web.bind.annotation.RequestMethod.GET;
#RestController
#RequestMapping("/log")
public class LogSubscription {
private static final Logger log = Logger.getLogger(LogSubscription.class.getName());
#RequestMapping(method = GET)
public String logSomething() {
log.log(Level.INFO, "Should see this in the console");
log.log(Level.SEVERE, "This is severe");
log.info("Normal Log Message");
return "Should log successfully";
}
}
Logging to the console works perfectly fine when I run locally. I just can't see the logs in the web console. I can see the GET request but not the logging in it. I am attaching a screenshot of my log.
I was finally able to find a solution. You must add the following to the build.gradle file:
configurations {
compile.exclude group: "org.slf4j", module: "jul-to-slf4j"
}

Executing Template From App Engine using Java

I have to Trigger a Template of dataflow Job from App Engine in Fixed Interval of Time to Make Fix Interval I Have used cron job but do not have any Idea how to trigger a Template in Java Language I Need the Below Code in Form of Java.
import datetime
import logging
import os
from google.appengine.ext import ndb
import webapp2
from googleapiclient.discovery import build
from oauth2client.client import GoogleCredentials
class LaunchJob(webapp2.RequestHandler):
credentials = GoogleCredentials.get_application_default()
service = build('dataflow', 'v1b3', credentials=credentials)
# Set the following variables to your values.
JOBNAME = 'kiss-fn-dataflow-job'
PROJECT = 'testing1-18001111'
BUCKET = 'kiss-bucket'
TEMPLATE = 'Test1'
GCSPATH="gs://{bucket}/templates/{template}".format(bucket=BUCKET, template=TEMPLATE),
BODY = {
"jobName": "{jobname}".format(jobname=JOBNAME),
"parameters": {
"inputFile" : "gs://{bucket}/input/my_input.txt",
"outputFile": "gs://{bucket}/output/my_output".format(bucket=BUCKET)
},
"environment": {
"tempLocation": "gs://{bucket}/temp".format(bucket=BUCKET),
"zone": "us-central1-f"
}
}
request = service.projects().templates().launch(projectId=PROJECT, gcsPath=GCSPATH, body=BODY)
response = request.execute()
app = webapp2.WSGIApplication([
('/', LaunchJob),
], debug=True)
The Above Program Runs Perfectly But to Deploy a Single application Many Dependency in Python is Done and Some Functionality is Not Available as Per Requirement to I need to Change my App Engine Program in Java. So I Can Usee Apache beam in My App.

Google App Engine does not recognized #Named parameter

I am using the Google Plugin for Eclipse, and I am writing an App Engine app as a Dynamic Web Module in Eclipse WTP.
I have defined the following Java class to serve as a Cloud Endpoint API:
package mypackage;
import static mypackage.OfyService.ofy;
import java.util.List;
import java.util.logging.Logger;
import mypackage.models.ProbeEntry;
import mypackage.models.ProbeSet;
import com.google.api.server.spi.config.Api;
import com.google.api.server.spi.config.ApiMethod;
import com.google.api.server.spi.config.ApiNamespace;
import com.google.api.server.spi.config.Named;
import com.googlecode.objectify.ObjectifyService;
#Api(name = "analysisEndpoint",
version = "v1",
namespace = #ApiNamespace(
ownerDomain = "myorg",
ownerName = "myorg",
packagePath = "analysis")
)
public class AnalysisEndpoint {
private static final Logger logger = Logger.getLogger(AnalysisEndpoint.class.getName());
#ApiMethod(name = "getMyProbeEntries", httpMethod = ApiMethod.HttpMethod.GET)
public ProbeSet getMyProbeEntries(#Named("amount") int amount) {
ObjectifyService.begin();
List<ProbeEntry> probeList = ofy().load().type(ProbeEntry.class).limit(amount).list();
return new ProbeSet(probeList);
}
}
I attempt to deploy to the Google App Engine by right-clicking the project -> Google App Engine WTP -> Deploy Project to Remote Server. I see in my console that the project is compiling and uploading, but eventually errors out with:
99% Endpoints configuration not updated. The app returned an error when the Google Cloud Endpoints server attempted to communicate with it.
The error log on the app engine shows the following:
18:31:58.119
javax.servlet.ServletContext log: unavailable
com.google.api.server.spi.config.validation.MissingParameterNameException: analysisEndpoint.myorg.analysis.AnalysisEndpoint.getMyProbeEntries parameter (type int): Missing parameter name. Parameter type (int) is not an entity type and thus should be annotated with #Named.
at
com.google.api.server.spi.config.validation.ApiConfigValidator.validateApiParameter(ApiConfigValidator.java:214)
...
As can be seen in the code, I do have #Named("amount") before the offending parameter. What is going wrong here? Side note: If I simply remove the amount parameter, the project deploys to App Engine without a problem.
Any help would be greatly appreciated.

My Google cloud endpoint API is not visible on the api explorer

I am very new to google app engine and endpoints and have been writing basic endpoint functions and deploying to the cloud. I succesfully deployed a HelloWorld endpoint and tested it over the API explorer: http://localhost:8080/_ah/api/explorer
But now when I have created a new endpoint API and followed the same steps (i.e deployed using new APP engine application name in the appengine-web.xml, run as appengine:update), the api explorer still shows my HelloWorld endpoint instead of my new API "yourfirstendpoint".
I've searched and tried to find an answer to no avail - and im sorry if this is a very basic and stupid question on my part (im sure it is) but i would realy appreciate if somebody could point me in the right direction on what i should be doing.
My API
package com.example.zinglife;
import com.google.api.server.spi.config.Api;
import com.google.api.server.spi.config.ApiMethod;
import com.google.api.server.spi.config.ApiMethod.HttpMethod;
import com.google.api.server.spi.response.NotFoundException;
import com.google.appengine.api.datastore.Key;
import com.google.appengine.api.datastore.KeyFactory;
/**
*
* Defines endpoint functions APIs.
*/
#Api(name = "yourfirstapi", version = "v1",
scopes = {Constants.EMAIL_SCOPE },
clientIds = {Constants.API_EXPLORER_CLIENT_ID},
description = "API for hello world endpoints.")
public class YourFirstAPI
{
#ApiMethod(name = "storeUserModel")
private User storeUserModel(User user) throws NotFoundException
{
String email = user.getEmail();
Key key = KeyFactory.createKey("User",email);
User userEntity = null;
try
{
if (userEntity==null)
{
userEntity = new User();
userEntity.setName(user.getName());
userEntity.setEmail(user.getEmail());
userEntity.setCountry(user.getCountry());
//
}
return userEntity;
}//*endtry
finally
{
}
}
}
The App engine Administrator Log after running the code:
Please let me know if any other information is needed :)
Make sure you have added your new service as one of the values for the 'services' parameter of the EndPointsServlet.
<servlet>
<!-- This is version 2.0 of the endpoints framework. -->
<servlet-name>EndpointsServlet</servlet-name>
<servlet-class>com.google.api.server.spi.EndpointsServlet</servlet-class>
<init-param>
<param-name>services</param-name>
<!-- Comma separated classes that provide endpoints -->
<param-value>
com.mycompany.myproduct.endpoint.SomeServiceV1,
com.mycompany.myproduct.endpoint.SomeServiceV2,
com.mycompany.myproduct.endpoint.SomeOtherServiceV1,
com.mycompany.myproduct.endpoint.SomeOtherServiceV2,
com.mycompany.myproduct.endpoint.SomeOtherServiceV3
</param-value>
</init-param>
</servlet>

Servlet mapping with annotation does not get resolved by Google App Engine Dev Server

Instead of servlet mapping in the web.xml, i'm trying to use annotation to map the servlet to urls as follows:
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
#WebServlet(name = "GuestbookServlet", urlPatterns = "/guestbook")
public class GuestbookServlet extends HttpServlet {
#Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
resp.setContentType("text/plain");
resp.getWriter().println("Hello, world");
}
}
And I have also declared the 3.0 spec for servlet in web.xml:
<?xml version="1.0" encoding="UTF-8"?>
<web-app
xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
version="3.0"
metadata-complete="false">
</web-app>
However, when I run it on my local environment the response returned is 404.
It works however if I just map the servlets in the web.xml. What am I doing wrong? Does GAE still not support 3.0 specs?
Servlet 3.0 spec is not supported by GAE/J
It's still on the roadmap: https://developers.google.com/appengine/docs/features#roadmap_features
You can star this issue to help show your support for this feature: https://code.google.com/p/googleappengine/issues/detail?id=3091
This ticket has been opened a lonnnggg time though.
When I run the following JSP code,
Server info == <%=application.getServerInfo()%><br/>
Major==<%=application.getMajorVersion()%><br/>
Minor==<%=application.getMinorVersion()%><br/>
JSP version is <%= JspFactory.getDefaultFactory().getEngineInfo().getSpecificationVersion()%><br/>
I see
Server info == Google App Engine/Google App Engine/1.8.1
Major==2
Minor==5
JSP version is 2.1
You could run it for yourself.
Servlet 3.1 are now supported on AppEngine.
... In addition to support for an updated JDK and Jetty 9 with Servlet 3.1 specs...
Announcement:
https://cloudplatform.googleblog.com/2017/06/Google-App-Engine-standard-now-supports-Java-8.html
Here is a code example

Resources