Error does not display image with the following url
http://127.0.0.1:8080/serve/CrObzPCoJfjG4ESUUb0hjw==
Image does exist in the blobstore can be checked in admin
My route
Dope on redirect routes
RedirectRoute('/serve/[a-zA-Z0-9-_]', handlers.ServeHandler, name='ServeHandler'),
My code:
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
import urllib
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
logging.info("SERVE " + str(resource))
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info)
class FetchHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file') # 'file' is file upload field in the form
blob_info = upload_files[0]
logging.info("FOUND blob info" + str(blob_info))
self.redirect('/serve/%s' % blob_info.key())
class ImageHandler(BaseHandler):
#user_required
def get(self, **kwargs):
user_session = self.user
user_session_object = self.auth.store.get_session(self.request)
upload_url = blobstore.create_upload_url('/fetch/')
user_info = models.User.get_by_id(long( self.user_id ))
user_info_object = self.auth.store.user_model.get_by_auth_token(
user_session['user_id'], user_session['token'])
try:
params = {
"upload_url": upload_url,
"user_session" : user_session,
"user_session_object" : user_session_object,
"user_info" : user_info,
"user_info_object" : user_info_object,
"userinfo_logout-url" : self.auth_config['logout_url'],
}
return self.render_template('image.html', **params)
except (AttributeError, KeyError), e:
return "Secure zone error:" + " %s." % e
I think your problem might be on this line:
self.redirect('/serve/%s' % blob_info.key())
According to the following recent changes assuming you did update appengine to the latest release:
The Blobstore service now returns the created filename instead of the blobKey when using Cloud Storage [link][1]
Have a look at the recent release notes and the changes that came with it.
I think they URL that you are providing to the create_upload_url is the wrong one, since you are defining it like /upload/.
Add the forward slash in the end and it should work:
upload_url = blobstore.create_upload_url('/upload/')
Comparing your code (form an earlier revision of your question) to some code I have working, I suspect that you might not want the trailing / on the /upload/ route (i.e., use /upload instead.
I'm not familiar with RedirectRoute, though.
Related
So.. This is my code for uploading an image with the Ferris2 frame work. Yay me, it works. However, see how I had to comment out gcs.open(... ? I don't want that commented out. I'd really love to just upload straight to cloud storage using that call, and not having to use anything related to blobs. What's the easiest way to accomplish this given that I'm stuck with using the AClassForm and the ferris framework?
class AClassForm(forms.model_form(AClass, exclude=('image_url') ) ):
image = FileField(u'Image File')
class AClasses(Controller):
class Meta:
Model = AClass
prefixes = ('admin',)
components = (scaffold.Scaffolding, Upload)
Form = AClassForm
admin_list = scaffold.list
admin_view = scaffold.view
admin_edit = scaffold.edit
admin_delete = scaffold.delete
def admin_add(self):
self.scaffold.ModelForm = AClassForm
self.scaffold.form_encoding = "multipart/form-data"
def before_save_callback(controller,container, item):
image = self.request.params["image"]
object_name = blobstore.parse_file_info(image).gs_object_name.split('/')[-1]
upload_settings = settings.get("upload")
url = upload_settings["url"]
bucket = upload_settings["bucket"]
#send to the cloud
#write a task to execute this?
item.image_url = url % (bucket, object_name)
#gcs_file= gcs.open("/".join(["", bucket, object_name]),
# 'w', content_type="image/jpeg",
#options={'x-goog-acl': 'public-read'} )
#gcs_file.write(item.image)#.file.encode('utf-8')) #
#gcs_file.close()
return
self.events.scaffold_before_save += before_save_callback
return scaffold.add(self)
I am not sure how Ferris works internally but you can use cloudstorage directly.
My image storage wrapper, provides resizing and returns a public URL to the upload for serving directly from storage.
import urlparse
from google.appengine.api import app_identity, blobstore, images
import cloudstorage
class ImageStorage(object):
def __init__(self, base_uri):
self.base_uri = "/{}/{}".format(app_identity.get_default_gcs_bucket_name(), base_uri.lstrip('/'))
def put(self, image, name, mime=None, width=None, height=None):
"""Puts an image into the Google Cloud Storage"""
if width or height:
image = images.resize(image, width=width or height, height=height or width)
mime = 'image/png' # resize defaults to output_encoding=PNG
options = {'x-goog-acl': 'public-read'}
with cloudstorage.open(self.make_key(name), 'w', content_type=mime, options=options) as fp:
fp.write(image)
return self.get_url(name)
def get_url(self, name):
"""Gets the url for an image from Google Cloud Storage"""
key = self.make_key(name)
# must be prefixed with /gs for the blob store to know it is from gcs
# https://cloud.google.com/appengine/docs/python/blobstore/#Python_Using_the_Blobstore_API_with_Google_Cloud_Storage
url = images.get_serving_url(blobstore.create_gs_key('/gs' + key))
# s/https/http/ if running under dev_appserver.py
# if not config.isLocal:
# parts = urlparse.urlparse(url)
# secure_parts = ('https',) + parts[1:]
# url = urlparse.urlunparse(secure_parts)
return url
def make_key(self, name):
"""Makes an item name key for Google Cloud Storage"""
return '%s/%s' % (self.base_uri, name)
Usage inside of a subclass of webapp2.RequestHandler. This stores the file with name "image" in the default bucket for your app in cloud storage at path /some/bucket/path/my-image-name.
thumb = self.request.POST["image"]
if hasattr(thumb, 'value'):
gs_base_uri = '/some/bucket/path'
image_storage = ImageStorage(gs_base_uri)
thumb_fn = 'my-image-name'
session_thumb_url = image_storage.put(image=thumb.value,
name=thumb_fn,
mime=thumb.type,
width=300, height=300)
return session_thumb_url
As I understand it, if you're using the Upload component of Ferris you can't escape the Blobstore, but the following comes pretty close. You don't have to use the Form class if you don't want to, I rarely use it myself. So imagine the following Controller:
from ferris import Controller, route
from ferris.components.upload import Upload
import cloudstorage as gcs
from google.appengine.ext import blobstore
import logging
class ImageManager(Controller):
class Meta:
components = (Upload,)
#route
def list(self):
#This just passes the upload URL to use in the form
self.context['upload_url'] = self.components.upload.generate_upload_url(uri=self.uri('image_manager:image_uploader_action'))
#route
def image_uploader_action(self):
# This gets all of the uploads passed in from the form
uploads = self.components.upload.get_uploads()
# This is the raw google cloud object reference. 'myfile' is the name of the upload field in the html form
file_gcs_obj_name = uploads['myfile'][0].cloud_storage.gs_object_name
# This is the blobstore key just for giggles
file_blobstore_key = uploads['myfile'][0].key()
# This will get rid of the preceeding junk you don't need i.e. "/gs/yadda/yadda"
clean_file_name = file_gcs_obj_name[3:]
# This is the name of the file as it was uploaded by the end-user
file_name_friendly = uploads['myfile'][0].filename
# This is the actual file, with this you can do whatever you want
the_actual_image = gcs.open(clean_file_name,'r')
# The file name by default is long and ugly, lets make a copy of the file with a more friendly name
new_filename = '/mydomain.appspot.com/'+file_name_friendly
gcs.copy2(clean_file_name,new_filename)
# We can generate a serving URL by using the blobstore API
valid_blob_reference = blobstore.create_gs_key('/gs'+new_filename)
file_serving_url = images.get_serving_url(valid_blob_reference)
logging.info('the serving url is: %s'% file_serving_url)
# delete the original image from cloud storage
gcs.delete(clean_file_name)
# Delete the original image from blobstore
blobstore.delete(file_blobstore_key)
# Close the file
the_actual_image.close()
return 'Done. go check the cloud storage browser'
Now all you need is the HTML form. You can use something like this:
{% extends "layouts/default.html" %}
{% block layout_content %}
<form name="myform" action="{{upload_url}}" method="POST" enctype="multipart/form-data">
<input type="file" name="myfile" id="fileToUpload">
<input type="submit" value="Upload File" name="submit">
</form>
{% endblock %}
Ferris is still going to place a file in the blobstore but you can delete it after you've used the cloudstorage.copy2() function. That function is fairly new so remember to update your cloudstorage package, you can download the latest copy from Google or Pypi (https://pypi.python.org/pypi/GoogleAppEngineCloudStorageClient/1.9.22.1)
Hope this helps.
I'd like to avoid using Webapp from GAE, so i use this code to upload an image to the Blobstore (code snippet from : http://flask.pocoo.org/mailinglist/archive/2011/1/8/app-engine-blobstore/#7fd7aa9a5c82a6d2bf78ccd25084ac3b)
#app.route("/upload", methods=['POST'])
def upload():
if request.method == 'POST':
f = request.files['file']
header = f.headers['Content-Type']
parsed_header = parse_options_header(header)
blob_key = parsed_header[1]['blob-key']
return blob_key
It returns what it seems to be indeed a Blobkey, wich is something like this :
2I9oX6J0U5nBCVw8kEndpw==
I then try to display the recently stored Blob image with this code :
#app.route("/testimgdisplay")
def test_img_display():
response = make_response(db.get("2I9oX6J0U5nBCVw8kEndpw=="))
response.headers['Content-Type'] = 'image/png'
return response
Sadly this part doesn't work, I got the following error :
BadKeyError: Invalid string key 2I9oX6J0U5nBCVw8kEndpw==
Do you guys have faced this error before ? It seems the Blobkey is well-formatted, and I can't find a clue.
There was a simple mistake on the call for getting the Blob, I wrote:
db.get("2I9oX6J0U5nBCVw8kEndpw==")
and the right call was instead:
blobstore.get("2I9oX6J0U5nBCVw8kEndpw==")
For those looking for a complete Upload/Serving image via GAE Blobstore and Flask without using Webapp, here is the complete code:
Render the template for the upload form:
#app.route("/upload")
def upload():
uploadUri = blobstore.create_upload_url('/submit')
return render_template('upload.html', uploadUri=uploadUri)
Place your uploadUri in the form path (html):
<form action="{{ uploadUri }}" method="POST" enctype="multipart/form-data">
Here is the function to handle the upload of the image (I return the blob_key for practical reasons, replace it with your template):
#app.route("/submit", methods=['POST'])
def submit():
if request.method == 'POST':
f = request.files['file']
header = f.headers['Content-Type']
parsed_header = parse_options_header(header)
blob_key = parsed_header[1]['blob-key']
return blob_key
Now say you serve your images with a path like this:
/img/imagefilename
Then your image serving function is :
#app.route("/img/<bkey>")
def img(bkey):
blob_info = blobstore.get(bkey)
response = make_response(blob_info.open().read())
response.headers['Content-Type'] = blob_info.content_type
return response
Finally, anywhere you need to display an image in a template, you simply put the code:
<img src="/img/{{ bkey }} />
I don't think Flask is any better or worse than Webapp in serving up Blobstore images, since they both use the Blobstore API for Serving a Blob.
What you're calling a Blobkey is just a string, which needs to be converted into a key (called resource here):
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info)
I am working on an app to process emails hitting my mailbox. I have modified my mail settings to forward incoming mails to myapp. The mails reaching myapp will be routed to the handler script ("handle_incoming_email.py") where it will be processed. My app.yaml file looks like this
app.yaml
application: myapp
version: 1-1
runtime: python27
api_version: 1
threadsafe: false
default_expiration: "360d"
handlers:
- url: /_ah/mail/.+
script: myapp/utils/handle_incoming_email.py
The mail handler script is given below
handle_incoming_email.py
import logging
import urllib
import base64
import traceback
from google.appengine.ext import webapp
from google.appengine.ext import blobstore
from google.appengine.api import urlfetch
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext.webapp.mail_handlers import InboundMailHandler
class ReceiveEmail(InboundMailHandler):
def receive(self, message):
try:
for filename, filecontents in message.attachments:
if filecontents.encoding:
# data = filecontents
data = filecontents.decode()
# data = filecontents.payload
# data = filecontents.payload.decode()
# data = base64.b64decode(filecontents.decode())
# data = base64.b64decode(filecontents.payload)
# data = base64.b64decode(filecontents.payload.decode())
upload_url = blobstore.create_upload_url('http://myapp.appspot.com/settings/saveItem/')
form_fields = {'field_name': data}
form_data = urllib.urlencode(form_fields)
result = urlfetch.fetch(url=upload_url,
payload=form_data,
method=urlfetch.POST,
headers={'Content-Type': 'application/x-www-form-urlencoded'})
logging.info(result)
except Exception, e:
traceback.print_exc()
application = webapp.WSGIApplication([ReceiveEmail.mapping()], debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
My requirement is to create an entity corresponding to each mail with attachment. For this, I need to parse the attachment from the mail and upload it to blobstore. However, when I try to upload the attachment to blobstore, I get the following error:
The request's content type is not accepted on this URL.
As you can see in the commented code in "handle_incoming_email.py", I tried different methods (trial-and-error) to get the data correct, but to no avail.
Could someone guide me in fixing this!
Thanks!!!
I think this code sample will solve you problem. You probably need to use only encode_multipart_formdata function from this code. And do not forget to set properly content type.
class BlobstoreUpload(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file')
blob_info = upload_files[0]
return self.response.write(blob_info.key())
#classmethod
def encode_multipart_formdata(cls, fields, files, mimetype='image/png'):
"""
Args:
fields: A sequence of (name, value) elements for regular form fields.
files: A sequence of (name, filename, value) elements for data to be
uploaded as files.
Returns:
A sequence of (content_type, body) ready for urlfetch.
"""
boundary = 'paLp12Buasdasd40tcxAp97curasdaSt40bqweastfarcUNIQUE_STRING'
crlf = '\r\n'
line = []
for (key, value) in fields:
line.append('--' + boundary)
line.append('Content-Disposition: form-data; name="%s"' % key)
line.append('')
line.append(value)
for (key, filename, value) in files:
line.append('--' + boundary)
line.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename))
line.append('Content-Type: %s' % mimetype)
line.append('')
line.append(value)
line.append('--%s--' % boundary)
line.append('')
body = crlf.join(line)
content_type = 'multipart/form-data; boundary=%s' % boundary
return content_type, body
class UserProfile(webapp2.RequestHandler):
def post(self):
picture = self.request.POST.get('picture')
# Write new picture to blob
content_type, body = BlobstoreUpload.encode_multipart_formdata(
[], [('file', name, image)])
response = urlfetch.fetch(
url=blobstore.create_upload_url(self.uri_for('blobstore-upload')),
payload=body,
method=urlfetch.POST,
headers={'Content-Type': content_type},
deadline=30
)
blob_key = response.content
I am trying to create a simple web application using Google App Engine. I use jinja2 to render a frontend html file. User enters their AWS credentials and gets the output of regions and connected with them virtual machines.
I have a controller file, to which I import a model file and it looks like this:
import webapp2
import jinja2
import os
import model
jinja_environment = jinja2.Environment(loader=jinja2.FileSystemLoader(os.path.dirname(__file__)))
class MainPage(webapp2.RequestHandler):
def get(self):
template = jinja_environment.get_template('index.html')
self.response.out.write(template.render())
def request_a(self):
a = self.reguest.get('a')
return a
def request_b(self):
b = self.reguest.get('b')
return b
class Responce(webapp2.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.write(testing_ec2.me.get_machines())
app = webapp2.WSGIApplication([('/2', MainPage), ('/responce', Responce)], debug=True)
then I have a model file, to which I import controller file and it looks like this:
import boto.ec2
import controller
import sys
if not boto.config.has_section('Boto'):
boto.config.add_section('Boto')
boto.config.set('Boto', 'https_validate_certificates', 'False')
a = controller.MainPage.get()
b = controller.MainPage.get()
class VM(object):
def __init__(self, a, b):
self.a = a
self.b = b
self.regions = boto.ec2.regions(aws_access_key_id = a, aws_secret_access_key = b)
def get_machines(self):
self.region_to_instance = {}#dictionary, which matches regions and instances for this region
for region in self.regions:
conn = region.connect(aws_access_key_id = self.a, aws_secret_access_key = self.b)
reservations = conn.get_all_instances()#get reservations(need understand them better)
if len(reservations) > 0:#if there are reservations in this region
self.instances = [i for r in reservations for i in r.instances]#creates a list of instances for that region
self.region_to_instance[region.name] = self.instances
return self.region_to_instance
me = VM(a, b)
me.get_machines()
When I run this, it throws an error: type object 'MainPage' has no attribute 'request_a'
I assume, that it happens, because I do not call an instance of MainPage class and instead call a class itself. What is an instance of MainPage(and it`s parent webapp.RequestHandler) class? How do I call it inside another module?
Your code looks very strange to me. I do not understand your coding practice.
The general answer is : if you like to use methods of your MainPage, you can use inheritance.
But. If I understand the goal of your code. Why not call boto from your Responce class. But, here you use a get, where you should use a post, because you post a form with AWS credentials.
So I suggest :
create a MainPage with get and post methods to handle the form
in the the post method make the boto requests and send the result with jinja to the user.
See also Getting started with GAE Python27 and handling forms:
https://developers.google.com/appengine/docs/python/gettingstartedpython27/handlingforms?hl=nl
So I decided to rewrite my image gallery because of the new high performance image serving thing. That meant using Blobstore which I have never used before. It seemed simple enough until I tried to store the BlobKey in my model.
How on earth do I store reference to a blobstorekey in a Model? Should I use string or should I use some special property that I don't know about? I have this model
class Photo(db.Model):
date = db.DateTimeProperty(auto_now_add=True)
title = db.StringProperty()
blobkey = db.StringProperty()
photoalbum = db.ReferenceProperty(PhotoAlbum, collection_name='photos')
And I get this error: Property blobkey must be a str or unicode instance, not a BlobKey
Granted, I am a newbie in app engine but this is the first major wall I have hit yet.
Have googled around extensively without any success.
The following works for me. Note the class is blobstore.blobstore instead of just blobstore.
Model:
from google.appengine.ext.blobstore import blobstore
class Photo(db.Model):
imageblob = blobstore.BlobReferenceProperty()
Set the property:
from google.appengine.api import images
from google.appengine.api import blobstore
from google.appengine.ext.webapp import blobstore_handlers
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file') # 'file' is file upload field in the form
blob_info = upload_files[0]
entity = models.db.get(self.request.get('id'))
entity.imageblob = blob_info.key()
Get the property:
image_url = images.get_serving_url(str(photo.imageblob.key()))
Instead of a db.StringProperty() you need to use db.blobstore.BlobReferenceProperty (I think)
I'm still trying to figure this thing out as well, but thought I'd post some ideas.
Here are the reference pages from Google:
http://code.google.com/appengine/docs/python/datastore/typesandpropertyclasses.html
http://code.google.com/appengine/docs/python/datastore/typesandpropertyclasses.html#BlobReferenceProperty