I have to Trigger a Template of dataflow Job from App Engine in Fixed Interval of Time to Make Fix Interval I Have used cron job but do not have any Idea how to trigger a Template in Java Language I Need the Below Code in Form of Java.
import datetime
import logging
import os
from google.appengine.ext import ndb
import webapp2
from googleapiclient.discovery import build
from oauth2client.client import GoogleCredentials
class LaunchJob(webapp2.RequestHandler):
credentials = GoogleCredentials.get_application_default()
service = build('dataflow', 'v1b3', credentials=credentials)
# Set the following variables to your values.
JOBNAME = 'kiss-fn-dataflow-job'
PROJECT = 'testing1-18001111'
BUCKET = 'kiss-bucket'
TEMPLATE = 'Test1'
GCSPATH="gs://{bucket}/templates/{template}".format(bucket=BUCKET, template=TEMPLATE),
BODY = {
"jobName": "{jobname}".format(jobname=JOBNAME),
"parameters": {
"inputFile" : "gs://{bucket}/input/my_input.txt",
"outputFile": "gs://{bucket}/output/my_output".format(bucket=BUCKET)
},
"environment": {
"tempLocation": "gs://{bucket}/temp".format(bucket=BUCKET),
"zone": "us-central1-f"
}
}
request = service.projects().templates().launch(projectId=PROJECT, gcsPath=GCSPATH, body=BODY)
response = request.execute()
app = webapp2.WSGIApplication([
('/', LaunchJob),
], debug=True)
The Above Program Runs Perfectly But to Deploy a Single application Many Dependency in Python is Done and Some Functionality is Not Available as Per Requirement to I need to Change my App Engine Program in Java. So I Can Usee Apache beam in My App.
Related
I have this following Flask application/folder structure in python3.6 Enviroment:
site/
__init__.py
models.py
static/
templates/
dashboard/
index.html
login/
index.html
views/
__init__.py
login.py
config.py
run.py
My files looks like this:
run.py
from site import app
app.run()
config.py
class BaseConfig(object):
DEBUG = False
TESTING = False
WFT_CSRF_ENABLED = True
class DevConfig(BaseConfig):
DEBUG = True
SECRET_KEY = 'dev'
SQLALCHEMY_DATABASE_URI = 'DRIVER={ODBC Driver 11 for SQL Server};SERVER=myServer;DATABASE=myDB;TRUSTED_CONNECTION=yes'
SQLALCHEMY_TRACK_MODIFICATIONS = False
site/init.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config.from_object('config.DevConfig')
db = SQLAlchemy(app)
from site.views import login
app.register_blueprint(login.mod)
site/views/login.py
from flask import Blueprint, render_template
mod = Blueprint('login', __name__)
#mod.route('/')
def index():
return render_template('login/index.html')
site/models.py
class User(db.Model):
__table__ = db.Model.metadata.tables['users']
def __repr__(self):
return '<User %r>' % self.username
what i am trying to do is to connect and use a existing model in my ms-sql database which is called user.
I am very new with Flask, Flask-SQLAlchemy, Blueprint and Python and I am stucked on how to use the db.Model in my login.py. How to achieved this in a Blueprint paradigm?
Any help and direction will be much appriciated and thanks for the help in advanced!
regards,
maki
you can add the following lines to your login.py:
from site import db
from site.models import User
to add an new user to the database you can use this :
new_user = User()
db.session.add(new_user)
db.session.commit()
and add this in your user table :
_tablename__ = 'user'
you will need to change you database URI in the config by following the syntax suggested in the flask_sqlalchemy doc
replace what you have by this :
SQLALCHEMY_DATABASE_URI : mysql://username:password#server_adress/db_name
for user authentification i can suggest you flask_login
I am using the Google Plugin for Eclipse, and I am writing an App Engine app as a Dynamic Web Module in Eclipse WTP.
I have defined the following Java class to serve as a Cloud Endpoint API:
package mypackage;
import static mypackage.OfyService.ofy;
import java.util.List;
import java.util.logging.Logger;
import mypackage.models.ProbeEntry;
import mypackage.models.ProbeSet;
import com.google.api.server.spi.config.Api;
import com.google.api.server.spi.config.ApiMethod;
import com.google.api.server.spi.config.ApiNamespace;
import com.google.api.server.spi.config.Named;
import com.googlecode.objectify.ObjectifyService;
#Api(name = "analysisEndpoint",
version = "v1",
namespace = #ApiNamespace(
ownerDomain = "myorg",
ownerName = "myorg",
packagePath = "analysis")
)
public class AnalysisEndpoint {
private static final Logger logger = Logger.getLogger(AnalysisEndpoint.class.getName());
#ApiMethod(name = "getMyProbeEntries", httpMethod = ApiMethod.HttpMethod.GET)
public ProbeSet getMyProbeEntries(#Named("amount") int amount) {
ObjectifyService.begin();
List<ProbeEntry> probeList = ofy().load().type(ProbeEntry.class).limit(amount).list();
return new ProbeSet(probeList);
}
}
I attempt to deploy to the Google App Engine by right-clicking the project -> Google App Engine WTP -> Deploy Project to Remote Server. I see in my console that the project is compiling and uploading, but eventually errors out with:
99% Endpoints configuration not updated. The app returned an error when the Google Cloud Endpoints server attempted to communicate with it.
The error log on the app engine shows the following:
18:31:58.119
javax.servlet.ServletContext log: unavailable
com.google.api.server.spi.config.validation.MissingParameterNameException: analysisEndpoint.myorg.analysis.AnalysisEndpoint.getMyProbeEntries parameter (type int): Missing parameter name. Parameter type (int) is not an entity type and thus should be annotated with #Named.
at
com.google.api.server.spi.config.validation.ApiConfigValidator.validateApiParameter(ApiConfigValidator.java:214)
...
As can be seen in the code, I do have #Named("amount") before the offending parameter. What is going wrong here? Side note: If I simply remove the amount parameter, the project deploys to App Engine without a problem.
Any help would be greatly appreciated.
I am working with the Oauth2 Decorator provided by Google. Right now I am just trying to do a very simple login via Oauth2 to Google using GAE. I am running locally for test purposes and have been successful in authenticating with Google; however, prior to the Google screen for authentication it always presents me with a local login screen running on localhost (//localhost:14080/_ah/login?continue=http%3A//localhost%3A14080/). I am not sure why I am getting this local login screen which does not appear to have any bearing on the Google login screen that comes after. I am wondering how to avoid this local login screen? Very simple code for test purposes:
import webapp2
import jinja2
from apiclient.discovery import build
from google.appengine.api import users
from oauth2client.appengine import OAuth2Decorator
template_dir = os.path.join(os.path.dirname(__file__), "templates")
jinja_env = jinja2.Environment(loader = jinja2.FileSystemLoader(template_dir))
decorator = OAuth2Decorator(
client_id='the id given by google',
client_secret='the secret given by google',
scope='https://www.googleapis.com/auth/userinfo.profile https://www.googleapis.com/auth/userinfo.email')
class Handler(webapp2.RequestHandler):
def write(self, *a, **kw):
self.response.out.write(*a, **kw)
def render_str(self, template, **params):
t = jinja_env.get_template(template)
return t.render(params)
def render(self, template, **kw):
self.write(self.render_str(template,**kw))
class MainHandler(Handler):
#decorator.oauth_required
def get(self):
service = build('oauth2', 'v2', http=decorator.http())
request = service.userinfo().get().execute()
self.write(request["email"])
app = webapp2.WSGIApplication([
('/', MainHandler),
(decorator.callback_path, decorator.callback_handler())
], debug=True)
The oauth2 decorator relies on having an appengine-logged-in user to function (it uses the user-id to store the oauth2 credentials), so without writing your own code, it isn't possible to avoid having the screen appear - in production, the login will be remembered for up to 30 days.
I have used lxml on Google App Engine to scrape some basic data.
It works fine with the SDK. When I try to use it on the appengine servers I get.
IOError: Error reading file 'http://www.google.com': failed to load external entity "http://www.google.com"
My code looks like;
import lxml.html
url = "http://www.google.com"
t = lxml.html.parse(url)
pagetitle = t.find.(".//title").text
self.response.out.write(pagetitle)
edit:
I ended up having to make a small change to handle as is outlined in the answer below.
from google.appengine.api import urlfetch
result = urlfetch.fetch(url)
t = lxml.html.fromstring(result.content)
GAE does not support opening sockets, you should use urlfetch.fetch() to get the page contents, then feed it to the parser.
I'm learning developing in Google App Engine.
This is one of the code from the tutorial, http://code.google.com/appengine/docs/python/gettingstarted/usingwebapp.html
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
class MainPage(webapp.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Hello, webapp World!')
application = webapp.WSGIApplication(
[('/', MainPage)],
debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
I've almost identical code. I sometime get warning:
WARNING 2011-06-30 13:10:44,443 init.py:851] You are using the default Django version (0.96). The default Django version will change in an App Engine release in the near future. Please call use_library() to explicitly select a Django version. For more information see http://code.google.com/appengine/docs/python/tools/libraries.html#Django
Can anyone please re factor the above code with use_library(). I'm not sure how to start and where to use use_library and what to do with webapp.
Thanks in advance.
The above code should not require you to call use_library directly.
If you create a new file in the root directory of your application named appengine_config.py and add the following line to it:
# Make webapp.template use django 1.2
webapp_django_version = '1.2'
try putting this code on top of your module :
import os
from google.appengine.dist import use_library
use_library('django', '1.2')
In the current version this is even simpler as third-party libraries are now specified in app.yaml
libraries:
- name: django
version: "1.2"
You can also use webapp2 that includes Django’s templating engine.
import webapp2
from google.appengine.ext.webapp2 import template