Unittest Jinja2 and Webapp2 : template not found - google-app-engine

I use Jinja2 with Webapp2 on a GAE project.
I have a base RequestHandler as describe in webapp2_extras.jinja2:
import webapp2
from webapp2_extras import jinja2
def jinja2_factory(app):
"""Set configuration environment for Jinja."""
config = {my config...}
j = jinja2.Jinja2(app, config=config)
return j
class BaseHandler(webapp2.RequestHandler):
#webapp2.cached_property
def jinja2(self):
# Returns a Jinja2 renderer cached in the app registry.
return jinja2.get_jinja2(factory=jinja2_factory, app=self.app)
def render_response(self, _template, **context):
# Renders a template and writes the result to the response.
rv = self.jinja2.render_template(_template, **context)
self.response.write(rv)
And a view handler as:
class MyHandler(BaseHandler):
def get(self):
context = {'message': 'Hello, world!'}
self.render_response('my_template.html', **context)
My templates are in the default location (templates).
The app works well on dev server, and the template is correctly rendered.
But when I try to unittest MyHandler with
import unittest
import webapp2
import webstest
class MyHandlerTest(unittest.TestCase):
def setUp(self):
application = webapp2.WSGIApplication([('/', MyHandler)])
self.testapp = webtest.TestApp(application)
def test_response(self):
response = application.get_response('/')
...
application.get_response('/my-view') raise an exception: TemplateNotFound: my_template.html.
Is there something I missed? Like a jinja2 environment or template loader configuration?

Problem origin:
Jinja2 default loader searches files in a relative ./templates/ directory. When you run your GAE application on the development server this path is relative to the root of your application. But when you run your unittests this path is relative to your unittest files.
Solution:
Not really an ideal solution, but here a trick I did to solve my problem.
I updated the jinja2 factory to add a dynamic template path, set in app config:
def jinja2_factory(app):
"""Set configuration environment for Jinja."""
config = {'template_path': app.config.get('templates_path', 'templates'),}
j = jinja2.Jinja2(app, config=config)
return j
And I set an absolute path to the templates in the setUp of my unittests:
class MyHandlerTest(unittest.TestCase):
def setUp(self):
# Set template path for loader
start = os.path.dirname(__file__)
rel_path = os.path.join(start, '../../templates') # Path to my template
abs_path = os.path.realpath(rel_path)
application.config.update({'templates_path': abs_path})

Related

Python webapp Google app engine: no module named six

The following code should take 2 dates from input.html and display months and years in output.html, but the Google app engine returns an error saying module six is missing, even though I have added all the site packages in my project library.
What am I doing wrong?
import webapp2
import jinja2
import os
import time
import datetime
import sys
from dateutil.rrule import rrule, MONTHLY
sys.path.append(os.path.join(os.path.dirname(__file__), "libs"))
template_dir = os.path.join(os.path.dirname(__file__), 'templates')
jinja_env = jinja2.Environment(loader = jinja2.FileSystemLoader(template_dir),
autoescape = True)
def render_str(template, **params):
t = jinja_env.get_template(template)
return t.render(params)
class MainHandler(webapp2.RequestHandler):
def write(self, *a, **kw):
self.response.out.write(*a, **kw)
def render_str(self, template, **params):
return render_str(template, **params)
def render(self, template, **kw):
self.write(self.render_str(template, **kw))
def get(self):
self.render('input.html')
def post(self):
frmstring=self.request.get('from')
tostring=self.request.get('to')
frm=time.strptime(frms,"%Y-%m")
to=time.strptime(tos,"%Y-%m")
dates = [dt for dt in rrule(MONTHLY, dtstart=frm, until=to)]
months_choices = []
for i in range(1,13):
months_choices.append(datetime.date(2008, i, 1).strftime('%B'))
self.render('output.html',dates = dates,months_choices=months_choices)
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)
You need to vendor six into your environment.
Run
pip install -t lib six
Then add these lines to your apppengine_config.py file (or create it)
# appengine_config.py
from google.appengine.ext import vendor
# Add any libraries install in the "lib" folder.
vendor.add('lib')
See https://cloud.google.com/appengine/docs/standard/python/tools/using-libraries-python-27#installing_a_third-party_library for more details
The six module is not provided by the AppEngine environment. Have you included it in your lib/ directory?

Catch All Script in App Engine Python (APP.YAML) does not work in Static Files

I have tried everything but it seems that you cannot get a catch all url...
- url: /.*
script: not_found.py
...to work on urls that are based on static directory paths. eg. I can type in www.foobar.com/asdas/asd/asd/asd/ad/sa/das/d and I can get a nice custom 404 page. But if I alter a static path url like www.foobar.com/mydir/mydir/mypage.html, I just get the horrible generic 404....
Error: Not Found
The requested URL /mydir/mydir/mypage.html was not found on this server.
... I would like to alter whatever catches the url in directory paths and writes the 404. This appears the only way to get a consistent custom 404 page in GAE Python.
Can anyone help? I have written my website from scratch and have a very limited knowledge of Python. Achieving a consistent custom 404 is the only thing I cannot seem to overcome.
EDIT/ADD : OK I've added the kind suggestion of #Lipis , and gone through getting started which which thankfully has given me a much better understanding of classes (I sadly can't vote it up yet). But! I am using a .py script found on the net and I think the NotFound class is interfering with the class that gives my index page, because now my index page is the 404 page specified by the Jinja! I have very little understanding of MainHandler so I may have to give up for now.
import os
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
from google.appengine.ext.webapp import template
from google.appengine.ext.webapp.util import run_wsgi_app
import jinja2
class MainHandler(webapp.RequestHandler):
def get (self, q):
if q is None:
q = 'index.html'
path = os.path.join (os.path.dirname (__file__), q)
self.response.headers ['Content-Type'] = 'text/html'
self.response.out.write (template.render (path, {}))
class NotFound(webapp.RequestHandler):
def post(self):
# you need to create the not_found.html file
# check Using Templates from Getting Started for more
jinja_environment = jinja2.Environment(
loader=jinja2.FileSystemLoader(os.path.dirname(__file__)))
template = jinja_environment.get_template('404.html')
self.response.out.write(template.render(template_values))
def main ():
application = webapp.WSGIApplication ([('/(.*html)?', MainHandler),('/.*', NotFound)],
debug=True)
util.run_wsgi_app (application)
if __name__ == '__main__':
main ()
For better understanding I'll make some modifications on the Getting Started example which I assume that you have done it and you made some experiments with it.
It's not a good idea to have the static file for all the not found pages in the app.yaml since most likely you would like to show something more dynamic and usually the - url: /.* should be handled within your app.
In this example we are going to add a new RequestHandler for all your not found pages
import jinja2
import os
# more imports
jinja_environment = jinja2.Environment(loader=jinja2.FileSystemLoader(os.path.dirname(__file__)))
class MainPage(webapp2.RequestHandler):
def get(self):
template = jinja_environment.get_template('index.html')
self.response.out.write(template.render(template_values))
class NotFound(webapp.RequestHandler):
def get(self):
# you need to create the not_found.html file
# check Using Templates from Getting Started for more
template = jinja_environment.get_template('not_found.html')
self.response.out.write(template.render(template_values))
application = webapp.WSGIApplication(
[('/', MainPage),
('/.*', NotFound)], # <-- This line is important
debug=True)
But in order to make the jinja2 templates work, follow carefully the modifications that you need to do in Using Templates section from the Getting Started.
The order in the URL mapping is very important so this catch all regular expression (/.*) should be always the last one, because otherwise all the other rules will be skipped.
If you want to catch all URLs, you will have to modify your main request handler in your file "not_found.py" by adding '/.*'.
For example, you can set the file "not_found.py" to:
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write("Hello, MAIN!")
application = webapp.WSGIApplication(
[('/.*', MainHandler)], # <--- Add '/.*' here
debug=True)
def main():
run_wsgi_app(application)
If you navigate to www.foobar.com/asd/ad/sa/das/d or any other URL, you will see the message "Hello, MAIN!.
Hope it helps. Ask question if needed

Prototyping: Simplest HTTP server with URL routing (to use w/ Backbone.Router)?

We're working on a Backbone.js application and the fact that we can start a HTTP server by typing python -m SimpleHTTPServer is brilliant.
We'd like the ability to route any URL (e.g. localhost:8000/path/to/something) to our index.html so that we can test Backbone.Router with HTML5 pushState.
What is the most painless way to accomplish that? (For the purpose of quick prototyping)
Just use the built in python functionality in BaseHTTPServer
import BaseHTTPServer
class Handler( BaseHTTPServer.BaseHTTPRequestHandler ):
def do_GET( self ):
self.send_response(200)
self.send_header( 'Content-type', 'text/html' )
self.end_headers()
self.wfile.write( open('index.html').read() )
httpd = BaseHTTPServer.HTTPServer( ('127.0.0.1', 8000), Handler )
httpd.serve_forever()
Download and install CherryPy
Create the following python script (call it always_index.py or something like that) and also replace 'c:\index.html' with the path of your actual file that you want to use
import cherrypy
class Root:
def __init__(self, content):
self.content = content
def default(self, *args):
return self.content
default.exposed = True
cherrypy.quickstart(Root(open('c:\index.html', 'r').read()))
Run python <path\to\always_index.py>
Point your browser at http://localhost:8080 and no matter what url you request, you always get the same content.

How do I fix this TypeError?

I'm new to python and Google App Engine. I'm trying to refactor this code from Nick Johnson blog to use webapp2 and python 2.7. http://blog.notdot.net/2009/10/Blogging-on-App-Engine-part-1-Static-serving
Anyways, when I run the code below I get this error.
TypeError: get() takes exactly 2 arguments (1 given)
I think it may have something to do with the path variable not being defined, but I don't know how to define it.
import webapp2
from google.appengine.ext import webapp
from google.appengine.ext import db
class StaticContent(db.Model):
body = db.BlobProperty()
content_type = db.StringProperty(required=True)
last_modified = db.DateTimeProperty(required=True, auto_now=True)
def get(path):
return StaticContent.get_by_key_name(path)
def set(path, body, content_type, **kwargs):
content = StaticContent(
key_name=path,
body=body,
content_type=content_type,
**kwargs)
content.put()
return content
class MainHandler(webapp2.RequestHandler):
def get(self, path):
content = get(path)
if not content:
self.error(404)
return
app = webapp2.WSGIApplication([('/', MainHandler)],
debug=True)
The error is raised because the get method of the MainHandler class expects a path parameter.
You should add grouping to the regex in your routing definition to pass the path parameter to the get method:
app = webapp2.WSGIApplication([('(/.*)', MainHandler)],
debug=True)

Appengine serve gzipped files

I'm using AppEngine to store some pickled python objects in my app. I want to serve these to the user directly, and I'm simply using the X-AppEngine-Blobkey header to serve the files to the user with a file.pickle.gz filename. However, when I try to extract these on my computer (Mac OS) using a simple double click, the files are turned into file.pickle.gz.cpgz.
I thought it was my browser being sneaky and extracting them, but I don't think so, since
pickle.load('file.pickle.gz')
Doesn't work, and neither does
pickle.load('file.pickle.gz.cpgz')
To store the files, I use:
blobfile = files.blobstore.create(mime_type='application/gzip')
with files.open(blobfile, 'a') as f:
gz = gzip.GzipFile(fileobj=f,mode='wb')
gz.write(my_pickled_object)
gz.close()
files.finalize(blobfile)
I think I'm not understanding the way gzips work. Can someone explain?
Are you sure file.pickle.gz.cpgz is the result of your double-clicking on the file.pickle.gz file you downloaded? Usually ".cpgz" is a different kind of archive file.
I can get the code you posted to work in a development server without significant changes. Here's the code, if it helps:
#!/usr/bin/env python
from __future__ import with_statement
import gzip
import pickle
from google.appengine.api import files
from google.appengine.api import memcache
from google.appengine.ext import blobstore
from google.appengine.ext import webapp
from google.appengine.ext.webapp import blobstore_handlers
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write('Hello world! make get')
class MakeFileHandler(webapp.RequestHandler):
def get(self):
data = pickle.dumps({'a':1, 'b':True, 'c':None})
blobfile = files.blobstore.create(mime_type='application/gzip')
with files.open(blobfile, 'a') as f:
gz = gzip.GzipFile(fileobj=f,mode='wb')
gz.write(data)
gz.close()
files.finalize(blobfile)
memcache.set('filekey', files.blobstore.get_blob_key(blobfile))
self.redirect('/')
class GetFileHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self):
blobkey = memcache.get('filekey')
if blobkey:
self.send_blob(blobkey)
else:
self.response.out.write('No data key set back')
def main():
application = webapp.WSGIApplication([('/', MainHandler),
('/make', MakeFileHandler),
('/get', GetFileHandler)],
debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
Click on "make", then click on "get". A file named "get.gz" is downloaded to your ~/Downloads/ folder (at least in Chrome). Double-click on it to produce a file named "get". Then:
% python
>>> import pickle
>>> pickle.load(open('get'))
{'a': 1, 'c': None, 'b': True}

Resources