Compiled templates with macros do not work on app engine - google-app-engine

I use Jinja2 compiled templates and a module loader to load the compiled templates (python code) from the datastore. But when my template contains a macro it does not work on app engine: TypeError: 'NoneType' object is not callable
But in the app engine SDK it works fine.
And when I skip the macro call, i receive the same error.
Without the macro it works fine. Without a solution for this macro problem, I call a python function in my template to implement the functionality of the macro.
Update: This is the template source code which results in an error:
{% extends "mainpage.html" %}
{% block form %}
{% macro test_macro(name) %}
<p>{{ name }}</p>
{% endmacro %}
<div>
{{ test_macro('John Doe') }}
</div>
{% endblock %}
And this is the compiled template code (the form block part):
def block_form(context, environment=environment):
if 0: yield None
yield u'\n'
def macro(l_name):
t_1 = []
pass
t_1.extend((
u'\n<p>',
to_string(l_name),
u'</p>\n',
))
return concat(t_1)
l_test_macro = Macro(environment, macro, 'test_macro', ('name',), (), False, False, False)
yield u'\n<div>\n\t%s\n</div>\n' % (
context.call(l_test_macro, 'John Doe'),
)
Update: After some debugging it worked. But I do understand it!!!
The problem: I lose my imports. And when I redefine my imports in the code. It worked.
The top of the module:
from __future__ import division
from jinja2.runtime import LoopContext, TemplateReference, Macro, Markup, TemplateRuntimeError, missing, concat, escape, markup_join, unicode_join, to_string, identity, TemplateNotFound
__jinja_template__ = None
To make it work I had to add an inline import:
from jinja2.runtime import Macro # import again ?????
l_test_macro = Macro(environment, macro, 'test_macro', ('name',), (), False, False, False)
Can somebody explain, how I can lose my imports ??? I have this problem only in app engine and NOT in the SDK ??? Is this a namespace problem?

I was able to solve it, by adding the module to the sys.modules. But I do not understand why it worked in the SDK and not in GAE when I use a macro
Here is the code of my changed module loader.
def get_module(self, environment, template):
# Convert the path to a module name
name = template.replace('.html', '').replace('.txt','').replace('/', '.') # NO extensions
module = None
if self.package == None : # load from db and not from package
logging.info('load module : ' + name) # load module from runtimes db
if name in sys.modules : return sys.modules[name] # already imported
try :
runtime = models.Runtimes.rtimes_get_by_key_name(template)
module_code = db.Text(runtime.compiled)
module = imp.new_module(name)
exec module_code in module.__dict__
sys.modules[name] = module # add to sys modules, so no import again
return module
except (ImportError, AttributeError):
logging.error('load failed : ' + name)
else : .... # load from package
raise TemplateNotFound(template)

Related

Build a react project in django

I am embedding a react project inside my django app. I am serving the index.html file created by npm run build from django as opposed to just using django as a third party api. I like this approach as it leverages django user authentication/ csrf tokens/ etc.
After npm run build I am extracting the head and the scripts on index.html and putting them in my base.html of django, this way, the react scripts will be available wherever I need them and I can use my django templates for my navbar, footer etc.
The problem I am having is I have to copy and paste the headers and scripts on every build, so my question is if there is a way to make npm run build build the files with the same name every time, or perhaps another solution, so when I rebuild the react project I don't have to recopy and paste to base.html?
Here is a code snippet sample
base.html
<html>
<head>
<!-- Copy and pasted header files from react build-->
</head>
<body>
<navbar></navbar>
{% block content %}
{% endblock %}
<script> //copy and pasted scripts from react build - different script names on every build </script>
</body>
</html>
And then a file served by django
homepage.html
{% extend 'base.html' %}
<div class="other-django-stuff"></div>
{% block content %}
<div id="root"></div> // where my react project is anchored
{% endblock %}
Well, so what is needed in this case is a post build script that allows you to edit the file called base.html to have the right names and scripts produced by the build. I will describe here some conceptual steps on how to achieve this result.
Step n. 1: Change the package.json file to allow the execution of a script after the build
In this case I am using a JavaScript script file that will be run using nodejs, but you can use also other languages or shell scripts
"scripts": {
...
"build": "react-scripts build && node ./postbuild.js",
...
},
Note that, since the && operator is used, the postbuild.js file will executed only if the build will have success.
Step n. 2: Create the post build script that will write your base.html file
With the script you will be able to write the base.html file dynamically.
Note:
Since you can create the script in different languages, the script itself it is not included in the answer.
Thanks to Emanuele's guidance Here is the following solution using python. This assumes the react static build folder is added to STATIC_DIRS in settings.py:
base.html
<html>
<head>
{% include 'react_head.html' %}
</head>
<body>
<navbar></navbar>
{% block content %}
{% endblock %}
{% include 'react_scripts.html' %}
<script src='django-scripts.js'></script>
</body>
</html>
build_base.py
from bs4 import BeautifulSoup
index = open( "build/index.html" )
soup = BeautifulSoup( index, features="html.parser" )
links = soup.findAll( "link" )
scripts = soup.findAll( "script" )
with open( "./../templates/react_head.html", "w" ) as html:
html.write( r"{% load static %}" + "\n" )
for link in links:
url = link["href"]
if "favicon" in url: continue
fname = "/".join( url.split("/static/")[1:] )
link["href"] = '{% static ' + f"'{fname}' " + r"%}"
html.write( link.prettify() )
with open( "./../templates/react_scripts.html", "w" ) as html:
html.write( r"{% load static %}" + "\n" )
for script in scripts:
if 'src' in script:
url = script["src"]
fname = "/".join( url.split("/static/")[1:] )
script["src"] = '{% static ' + f"'{fname}' " + r"%}"
html.write( script.prettify() )
package.json
"scripts": {
...
"build": "react-scripts build && python ./build_base.py",
...
},

how to fetch the frontmatter of any hugo markdown page (not just _index.md)

I'm trying to write a shortcode for my hugo site that fetches the title parameter of page.
I have a directory structure like this:
content
├── workshops
│   ├── foo
│   │   └── _index.md
│   ├── bar.md
This works perfectly:
{{ with .Site.GetPage "home" "workshops/foo"}}
{{ .Params.Title }}
{{ end }}
And this one consistently comes up blank (even though there is a title in the markdown).
{{ with .Site.GetPage "home" "workshops/bar"}}
{{ .Params.Title }}
{{ end }}
My question is: How so I get the title of a standalone page?
I've tried a bunch of different combinations of things and I'm just not coming right. I've tried reading the docs and they are horribly convoluted on this point.
I have a solution! I wrote a little Python3.7 script to make directories and move and rename markdown files and just ran it over my entire contents directory. This solved my problem but is a bit of a hack...
import logging
import os
from pathlib import Path
def fixup(path):
location = Path(path)
assert location.is_dir(), location
for child in location.iterdir():
if child.is_dir():
fixup(child)
else:
fix_file(child)
def fix_file(file_path):
name = file_path.name
if not name.endswith(".md"):
# we only care about markdown files.
return
check_metadata(file_path)
if name.startswith("_index."):
# looks good
return
# make a directory with the same name as the file (without the extension)
suffix = ''.join(file_path.suffixes)
prefix = name[: -len(suffix)]
new_dir = file_path.parent / prefix
new_dir.mkdir()
new_path = new_dir / f"_index{suffix}"
file_path.rename(new_path)
def check_metadata(file_path):
""" given the path to a markdown file, make sure that the frontmatter includes
the required metadata
"""
# TODO
# required = ['title']
# allowed = ['pre', 'weight', 'ready']
if __name__ == '__main__':
fixup('content')
Two differences:
Use the global site variable
Just pass the page name as the argument
{{ with site.GetPage "workshops/bar" }}
{{ .Title }}
{{ end }}

How to add a new hugo static page?

From the "getting started" section it seems this should work, but it doesn't.
hugo new site my-site
hugo new privacy.md
hugo server --watch --includeDrafts
curl -L localhost:1313/privacy/index.html
# 404 page not found
curl -L localhost:1313/privacy.html
# 404 page not found
curl -L localhost:1313/privacy/
# 404 page not found
How can I add a new page?
This is the best tutorial how to create static "landing pages" on Hugo: https://discuss.gohugo.io/t/creating-static-content-that-uses-partials/265/19?u=royston
Basically, you create .md in /content/ with type: "page" in front matter, then create custom layout for it, for example layout: "simple-static" in front matter, then create the layout template in themes/<name>/layouts/page/, for example, simple-static.html. Then, use all partials as usual, and call content from original .md file using {{ .Content }}.
All my static (landing) pages are using this method.
By the way, I'm not using hugo new, I just clone .md file or copy a template into /content/ and open it using my iA Writer text editor. But I'm not using Hugo server either, adapted npm-build-boilerplate is running the server and builds.
Just tested OK with this on Hugo 0.13:
hugo new site my-site
cd my-site
hugo new privacy.md
hugo server -w -D
curl -L localhost:1313/privacy/
Note: You have to either use a theme or provide your own layout template to get something more than a blank page. And of course, some Markdown in privacy.md would also make it even nicer.
See http://gohugo.io/overview/introduction for up-to-date documentation.
I had a similar requirement, to add static page (aboutus in this case). Following steps did the trick,
Created an empty file content/aboutus/_index.md
Created aboutus.html page layouts/section/aboutus.html
Make you have some default frontmatter set in archetypes/default.md
# archetypes/default.md
+++
title: "{{ replace .Name "-" " " | title }}"
date: {{ .Date }}
draft: true
+++
And the single layout in layouts/_default/single.html to render some variable or content
# tags to render markdown content
<h1>{{ .Title }}</h1>
<p>{{ .Content }}</p>
<span>{{ .Params.date }}</span>
Now type hugo new privacy.md which will create a new page on the following directory in content/privacy.md
Take "About" as example:
# will create content/about.md
hugo new about.md
Edit about.md and add the last 2 lines, the metadata/front matter looks like:
title: "About"
date: 2019-03-26
menu: "main"
weight: 50
That should work.

Render different static folder based on url request

Using Flask, I can render different templates based on the url. For example :
http://example.com/site1/ ; will render templates/site1/{htmls}
http://example.com/site2/ ; will render templates/site2/{htmls}
This works really great with jinja_loader and a custom Loader, but now I'm stuck with the static files.
The static files depends on the template, so they are located in templates/static/site{0-9}, but of course, I cannot set a jinja_loader on the static_folder parameter because it's not related to Jinja but to Flask.
How can I render the correct static folder based on the current URL?
As an example, here's the loaded code:
Flask(app_name,
static_url_path = '/public',
static_folder = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'templates/static')
)
and in templates/static, I have :
static/
site1/
css/
js/
images/
site2/
css/
js/
images/
etc...
You either have to use explicit paths in your static views:
url_for('static', filename='site1/css/...')
where site1 could be taken from request.path.split('/', 1)[0]:
url_for('static', filename='{}/css/...'.format(request.path.split('/', 1)[0]))
You could create a custom static view:
from flask import request, send_from_directory, current_app, safe_join
import os.path
#app.route('/<site>/static/<path:filename>')
def per_site_static(site, filename):
if site is None:
# pick site from the URL; only works if there is a `/site/` first element.
site = request.path.split('/')[0]
static_folder = safe_join(current_app.static_folder, site)
cache_timeout = current_app.get_send_file_max_age(filename)
return send_from_directory(static_folder, filename,
cache_timeout=cache_timeout)
Then use url_for() to generate urls:
{{ url_for('per_site_static', site=None, filename='css/...') }}
An other interesting alternative is the following :
class GenericStaticFlask(Flask):
def send_static_file(self, filename):
# g.site contains the name of the template path for siteX
return super(GenericStaticFlask, self).send_static_file("%s/%s" % (g.site, filename))
app = GenericStaticFlask(app_name,
static_url_path = '/public',
static_folder = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'templates/static')
)

Can't disable the autoescape in jinja2

In GAE I use jinja2 with the autoescape, and everything works well.
import jinja2
jinja_env = jinja2.Environment(loader = jinja2.FileSystemLoader(template_dir), autoescape = True)
In one template I don't want the autoescape, so I tried to disable it like this:
{% autoescape false %}
{{content}}
{% endautoescape %}
When it's time to render this template I get the message Encountered unknown tag 'autoescape'.
Try this:
{{ content | safe}}
docs:
Flask — Controlling Autoescaping
Jinja2 — Filters — safe
In order for the autoescape tag to be recognized, you need to enable the autoescape extension when setting up jinja2, like this:
jinja_env = jinja2.Environment(loader = jinja2.FileSystemLoader(template_dir),
autoescape = True,
extensions = ['jinja2.ext.autoescape'])
Also, make sure you're using jinja2 version 2.4 or higher in your app.yaml (the current version is GAE is 2.6):
libraries:
- name: jinja2
version: "2.6"
For more information, see the documentation for the autoescape extension.

Resources