I'm a newbie to GAE and am trying to work out how to host an RSS feed based on data stored on the service.
So far I can't work out an obvious way to do it having read through the docs.
Can anyone give me some pointers as to what API I should be using?
Why do you need an API? You can query for updated data using the normal query classes, and output the feed using the standard Python XML tools like ElementTree.
This isn't any different to serving a dynamic HTML page. Do it exactly the same way you would serve up an HTML template, but generate RSS or Atom instead of HTML.
The easiest way to get a quick solution is gae-rest but it is as old as 2008. You may find my solution helpful that renders my data to a GeoRSS template, so here's my solution:
<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xmlns:georss="http://www.georss.org/georss" xmlns:gml="http://www.opengis.net/gml">
<title>{{host}}</title>
<link href="http://{{host}}" rel="self"/>
<id>http://{{host}}/</id>
<updated>2011-09-17T08:14:49.875423Z</updated>
<generator uri="http://{{host}}/">{{host}}</generator>
{% for ad in ads %}
<entry>
<title><![CDATA[{{ad.title}}]]></title>
<link href="http://{{host}}/vi/{{ad.key.id}}"/>
<id>http://{{host}}/vi/{{ad.key.id}}</id>
<updated>{{ad.modified.isoformat}}Z</updated>
<author><name>{{ad.title|escape}}</name></author>
<georss:point>{{ad.geopt.lon|floatformat:2}},{{ad.geopt.lat|floatformat:2}}</georss:point>
<published>{{ad.added}}</published>
<summary type="xhtml"><div xmlns="http://www.w3.org/1999/xhtml">{{ad.text|escape}}</div>
</summary>
</entry>
{% endfor %}
</feed>
handler:
class GeoRSS(webapp2.RequestHandler):
def get(self):
start = datetime.datetime.now() - timedelta(days=60)
count = (int(self.request.get('count'
)) if not self.request.get('count') == '' else 1000)
try:
ads = memcache.get('ads')
except KeyError:
ads = Ad.all().filter('modified >',
start).filter('published =',
True).order('-modified').fetch(count)
memcache.set('ads', ads)
template_values = {'ads': ads, 'request': self.request,
'host': os.environ.get('HTTP_HOST',
os.environ['SERVER_NAME'])}
dispatch = 'templates/georss.html'
path = os.path.join(os.path.dirname(__file__), dispatch)
output = template.render(path, template_values)
self.response.headers["Cache-Control"] = "public,max-age=%s" % 86400
self.response.headers['Content-Type'] = 'application/rss+xml'
self.response.out.write(output)
Related
I'm trying to add search functionlity in the header of the website so it's global but I'm having an issue with the context part of Routable Pages because I don't know how to set it so that it always links to the HomePage model no matter the URL.
For example, this works on the homepage seeing as self is the homepage.
{% routablepageurl self 'post_search' %}
However, if I go to the blog page (localhost:8000/blog/) it doesn't work because "self" should be the homepage. So with my limited knowledge I understand what the issue is but have no idea how to go about solving it.
I did try:
{% routablepageurl homepage 'post_search' %}
but I get the error
'str' object has no attribute 'relative_url'
I know I could go through every Model and pass in the HomePage into the context but surely there is a better way to do this?
The reason why I have added this to the HomePage is because I want the URL to be domain.com/search/
class HomePage(RoutablePageMixin, Page):
template = 'lr/home/home_page.html'
def get_context(self, request, *args, **kwargs):
context = super().get_context(request, *args, **kwargs)
context['no_webpage_title'] = True
return context
class Meta:
verbose_name = 'R HomePage'
#route(r'^search/$', name='post_search')
def post_search(self, request, *args, **kwargs):
context = self.get_context(request)
search_query = request.GET.get('q', None)
self.posts = BlogDetailPage.objects.live().public().order_by('-last_published_at')
if search_query:
self.posts = self.posts.search(search_query)
print(self.posts)
return render(request, 'global/search_results_page.html', context)
Simple answer: If you've decided that your search URL will be /search/ and there will be no reason ever to change it, then don't bother trying to get {% routablepageurl %} to output an answer you already know - just write <form action="/search/">.
More complex answer: Implement your search form as an inclusion tag. This will allow you to write a template for the search form, along with a Python function to set up the variables you want to pass to that template. Within that Python function, you can retrieve the HomePage instance (probably with HomePage.objects.first()), call reverse_subpage on it to retrieve the search URL, and pass that to the template as part of the context.
So.. This is my code for uploading an image with the Ferris2 frame work. Yay me, it works. However, see how I had to comment out gcs.open(... ? I don't want that commented out. I'd really love to just upload straight to cloud storage using that call, and not having to use anything related to blobs. What's the easiest way to accomplish this given that I'm stuck with using the AClassForm and the ferris framework?
class AClassForm(forms.model_form(AClass, exclude=('image_url') ) ):
image = FileField(u'Image File')
class AClasses(Controller):
class Meta:
Model = AClass
prefixes = ('admin',)
components = (scaffold.Scaffolding, Upload)
Form = AClassForm
admin_list = scaffold.list
admin_view = scaffold.view
admin_edit = scaffold.edit
admin_delete = scaffold.delete
def admin_add(self):
self.scaffold.ModelForm = AClassForm
self.scaffold.form_encoding = "multipart/form-data"
def before_save_callback(controller,container, item):
image = self.request.params["image"]
object_name = blobstore.parse_file_info(image).gs_object_name.split('/')[-1]
upload_settings = settings.get("upload")
url = upload_settings["url"]
bucket = upload_settings["bucket"]
#send to the cloud
#write a task to execute this?
item.image_url = url % (bucket, object_name)
#gcs_file= gcs.open("/".join(["", bucket, object_name]),
# 'w', content_type="image/jpeg",
#options={'x-goog-acl': 'public-read'} )
#gcs_file.write(item.image)#.file.encode('utf-8')) #
#gcs_file.close()
return
self.events.scaffold_before_save += before_save_callback
return scaffold.add(self)
I am not sure how Ferris works internally but you can use cloudstorage directly.
My image storage wrapper, provides resizing and returns a public URL to the upload for serving directly from storage.
import urlparse
from google.appengine.api import app_identity, blobstore, images
import cloudstorage
class ImageStorage(object):
def __init__(self, base_uri):
self.base_uri = "/{}/{}".format(app_identity.get_default_gcs_bucket_name(), base_uri.lstrip('/'))
def put(self, image, name, mime=None, width=None, height=None):
"""Puts an image into the Google Cloud Storage"""
if width or height:
image = images.resize(image, width=width or height, height=height or width)
mime = 'image/png' # resize defaults to output_encoding=PNG
options = {'x-goog-acl': 'public-read'}
with cloudstorage.open(self.make_key(name), 'w', content_type=mime, options=options) as fp:
fp.write(image)
return self.get_url(name)
def get_url(self, name):
"""Gets the url for an image from Google Cloud Storage"""
key = self.make_key(name)
# must be prefixed with /gs for the blob store to know it is from gcs
# https://cloud.google.com/appengine/docs/python/blobstore/#Python_Using_the_Blobstore_API_with_Google_Cloud_Storage
url = images.get_serving_url(blobstore.create_gs_key('/gs' + key))
# s/https/http/ if running under dev_appserver.py
# if not config.isLocal:
# parts = urlparse.urlparse(url)
# secure_parts = ('https',) + parts[1:]
# url = urlparse.urlunparse(secure_parts)
return url
def make_key(self, name):
"""Makes an item name key for Google Cloud Storage"""
return '%s/%s' % (self.base_uri, name)
Usage inside of a subclass of webapp2.RequestHandler. This stores the file with name "image" in the default bucket for your app in cloud storage at path /some/bucket/path/my-image-name.
thumb = self.request.POST["image"]
if hasattr(thumb, 'value'):
gs_base_uri = '/some/bucket/path'
image_storage = ImageStorage(gs_base_uri)
thumb_fn = 'my-image-name'
session_thumb_url = image_storage.put(image=thumb.value,
name=thumb_fn,
mime=thumb.type,
width=300, height=300)
return session_thumb_url
As I understand it, if you're using the Upload component of Ferris you can't escape the Blobstore, but the following comes pretty close. You don't have to use the Form class if you don't want to, I rarely use it myself. So imagine the following Controller:
from ferris import Controller, route
from ferris.components.upload import Upload
import cloudstorage as gcs
from google.appengine.ext import blobstore
import logging
class ImageManager(Controller):
class Meta:
components = (Upload,)
#route
def list(self):
#This just passes the upload URL to use in the form
self.context['upload_url'] = self.components.upload.generate_upload_url(uri=self.uri('image_manager:image_uploader_action'))
#route
def image_uploader_action(self):
# This gets all of the uploads passed in from the form
uploads = self.components.upload.get_uploads()
# This is the raw google cloud object reference. 'myfile' is the name of the upload field in the html form
file_gcs_obj_name = uploads['myfile'][0].cloud_storage.gs_object_name
# This is the blobstore key just for giggles
file_blobstore_key = uploads['myfile'][0].key()
# This will get rid of the preceeding junk you don't need i.e. "/gs/yadda/yadda"
clean_file_name = file_gcs_obj_name[3:]
# This is the name of the file as it was uploaded by the end-user
file_name_friendly = uploads['myfile'][0].filename
# This is the actual file, with this you can do whatever you want
the_actual_image = gcs.open(clean_file_name,'r')
# The file name by default is long and ugly, lets make a copy of the file with a more friendly name
new_filename = '/mydomain.appspot.com/'+file_name_friendly
gcs.copy2(clean_file_name,new_filename)
# We can generate a serving URL by using the blobstore API
valid_blob_reference = blobstore.create_gs_key('/gs'+new_filename)
file_serving_url = images.get_serving_url(valid_blob_reference)
logging.info('the serving url is: %s'% file_serving_url)
# delete the original image from cloud storage
gcs.delete(clean_file_name)
# Delete the original image from blobstore
blobstore.delete(file_blobstore_key)
# Close the file
the_actual_image.close()
return 'Done. go check the cloud storage browser'
Now all you need is the HTML form. You can use something like this:
{% extends "layouts/default.html" %}
{% block layout_content %}
<form name="myform" action="{{upload_url}}" method="POST" enctype="multipart/form-data">
<input type="file" name="myfile" id="fileToUpload">
<input type="submit" value="Upload File" name="submit">
</form>
{% endblock %}
Ferris is still going to place a file in the blobstore but you can delete it after you've used the cloudstorage.copy2() function. That function is fairly new so remember to update your cloudstorage package, you can download the latest copy from Google or Pypi (https://pypi.python.org/pypi/GoogleAppEngineCloudStorageClient/1.9.22.1)
Hope this helps.
I am trying to get URL of Image(blob field of GAE):
class Product(db.Model):
name = db.StringProperty()
price = db.FloatProperty()
added = db.DateTimeProperty(auto_now_add=True)
image = db.BlobProperty(default=None)
url = images.get_serving_url(movie.image)
Handler of serve image:
def result(request):
product = Product()
product.name = "halva"
url = 'http://echealthinsurance.com/wp-content/uploads/2009/11/minnesota.jpg'
product.image = db.Blob(urlfetch.Fetch(url).content)
product.put()
template = loader.get_template("result.html")
context = RequestContext(request,
{
"result" : u"Add"})
return HttpResponse(template.render(context))
But i get except:
UnicodeDecodeError:
When try to ignore this exception(that was bug in Python 2.7) I get exception in other place.
And after that i try to encode Image to 'latin-1'('utf-8' don't work):
enc_img = movie.image.decode("latin-1")
url = images.get_serving_url(enc_img)
Result: url has a view like binary file:
"ÝêÓ9>èýÑNëCf Äàr0xã³3Ï^µ7±\íQÀ¡>.....ÕÝ£°Ëÿ"I¢¶L`ù¥ºûMþÒ¸ÿ+ÿL¢ï£ÿÙ' alt="" />"
How I get url to show dynamic image in template?
You are confusing two different things here.
If you are storing your image in a db.BlobProperty (code doesn't show you are doing this, but the Schema you have is using db.BlobProperty) this means your handler has to serve the image.
However you are using image.get_serving_url, which takes a BlobKey instance which comes from storing an Image in the BlobStore https://developers.google.com/appengine/docs/python/blobstore/blobkeyclass which is a completely different thing to what you are doing.
You will need to work out what you want to do, store an image (max size 1MB) in a BlobProperty and provide a handler that can serve the image, or upload it to the BlobStore and serve it from there
images.get_serving_url takes a BlobKey. Try:
enc_img = movie.image
url = images.get_serving_url(enc_img.key())
{% for result in results %}
{{ result.photo }}
{% endif %}
This clearly won't work, but I can't find any info for how to upload a photo. On the admin console, I can see that I have successfully uploaded the image to the blobstore, now how can I send it as to my webapp template?
I can show the description doing this.
{% for result in results %}
{{ result.description }}
{% endif %}
But I don't know how to get GAE to read the image file as an image.
Any help would be greatly appreciated.
Thanks everyone!
I have written a tutorial on this topic. Read it and if you have any specific problems post here.
http://verysimplescripts.blogspot.com/
You should have an <img> tag in your template that has a src attribute that contains a URL that is served by your application and delivers the data for the image. For example, let's say that you store images in a model called Image:
class Image(db.Model):
filename = db.StringProperty() # The name of the uploaded image
mime_type = db.StringProperty() # The mime type.
content = db.BlobProperty() # The bytes of the image
def load(id):
# Load an entity from the database using the id. Left as an
# exercise...
def link_id_for(self):
"Returns an id that uniquely identifies the image"
return self.key().id()
In the controller/request handler code that renders the page that contains the image, you would pass the id that is returned by link_id_for to the template, and the template would have your image tag, something like this:
<img src="/images/show/{{image_id}}">
You would have a request handler that handles requests to /images/show/id. You would use id to get the Image entity out of the datastore and the send it back in the response, something like this:
found_image = Image.load(id)
response.headers['Content-Type'] = str(found_image.mime_type)
response.out.write(found_image.content)
Obviously, you'd have to adapt the particulars of the code to your current application structure and conventions, but that's the heart of it: use an img tag with a src that points at your application; your application includes a request handler the delivers the bytes and a Content-Type header.
Currently I am using MySQLi to parse a CSV file into a Database, that step has been accomplished. However, My next step would be to make this Database searchable and automatically updated via jQuery.ajax().
Some people suggest that I print out the Database in another page and access it externally.
I'm quite new to jquery + ajax so if anyone could point me in the right direction that would be greatly appreciated.
I understand that the documentation on ajax should be enough to tell me what I'm looking for but it appears to talk only about retrieving data from an external file, what about from a mysql database?
The code so far stands:
<head>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4/jquery.min.js"></script>
</head>
<body>
<input type="text" id="search" name="search" />
<input type="submit" value="submit">
<?php
show_source(__FILE__);
error_reporting(E_ALL);ini_set('display_errors', '1');
$category = NULL;
$mc = new Memcache;
$mc->addServer('localhost','11211');
$sql = new mysqli('localhost', 'user', 'pword', 'db');
$cache = $mc->get("updated_DB");
$query = 'SELECT cat,name,web,kw FROM infoDB WHERE cat LIKE ? OR name LIKE ? OR web LIKE ? OR kw LIKE ?';
$results = $sql->prepare($query);
$results->bind_param('ssss', $query, $query, $query, $query);
$results->execute();
$results->store_result();
?>
</body>
</html>
I understand that the documentation on ajax should be enough to tell me what I'm looking for but it appears to talk only about retrieving data from an external file, what about from a mysql database?
Close. It fetches data from a URI. You need to provide a URI that the data can be requested from (so you need a server side script to get the data from the database and expose it over HTTP — you can't talk directly to the database from the browser).
You have got your data already, so you just need to write views for it.
Usually, people will write an HTML view first so they can build on something that works.
Then you just need to write an alternative view that generates data in a fashion that is easy to parse with JavaScript. JSON is popular, and PHP comes with facilities for generating JSON output.
jQuery will set an X-Requested-By header that you can use to choose between returning HTML or JSON output.