I'd like to avoid using Webapp from GAE, so i use this code to upload an image to the Blobstore (code snippet from : http://flask.pocoo.org/mailinglist/archive/2011/1/8/app-engine-blobstore/#7fd7aa9a5c82a6d2bf78ccd25084ac3b)
#app.route("/upload", methods=['POST'])
def upload():
if request.method == 'POST':
f = request.files['file']
header = f.headers['Content-Type']
parsed_header = parse_options_header(header)
blob_key = parsed_header[1]['blob-key']
return blob_key
It returns what it seems to be indeed a Blobkey, wich is something like this :
2I9oX6J0U5nBCVw8kEndpw==
I then try to display the recently stored Blob image with this code :
#app.route("/testimgdisplay")
def test_img_display():
response = make_response(db.get("2I9oX6J0U5nBCVw8kEndpw=="))
response.headers['Content-Type'] = 'image/png'
return response
Sadly this part doesn't work, I got the following error :
BadKeyError: Invalid string key 2I9oX6J0U5nBCVw8kEndpw==
Do you guys have faced this error before ? It seems the Blobkey is well-formatted, and I can't find a clue.
There was a simple mistake on the call for getting the Blob, I wrote:
db.get("2I9oX6J0U5nBCVw8kEndpw==")
and the right call was instead:
blobstore.get("2I9oX6J0U5nBCVw8kEndpw==")
For those looking for a complete Upload/Serving image via GAE Blobstore and Flask without using Webapp, here is the complete code:
Render the template for the upload form:
#app.route("/upload")
def upload():
uploadUri = blobstore.create_upload_url('/submit')
return render_template('upload.html', uploadUri=uploadUri)
Place your uploadUri in the form path (html):
<form action="{{ uploadUri }}" method="POST" enctype="multipart/form-data">
Here is the function to handle the upload of the image (I return the blob_key for practical reasons, replace it with your template):
#app.route("/submit", methods=['POST'])
def submit():
if request.method == 'POST':
f = request.files['file']
header = f.headers['Content-Type']
parsed_header = parse_options_header(header)
blob_key = parsed_header[1]['blob-key']
return blob_key
Now say you serve your images with a path like this:
/img/imagefilename
Then your image serving function is :
#app.route("/img/<bkey>")
def img(bkey):
blob_info = blobstore.get(bkey)
response = make_response(blob_info.open().read())
response.headers['Content-Type'] = blob_info.content_type
return response
Finally, anywhere you need to display an image in a template, you simply put the code:
<img src="/img/{{ bkey }} />
I don't think Flask is any better or worse than Webapp in serving up Blobstore images, since they both use the Blobstore API for Serving a Blob.
What you're calling a Blobkey is just a string, which needs to be converted into a key (called resource here):
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info)
Related
So.. This is my code for uploading an image with the Ferris2 frame work. Yay me, it works. However, see how I had to comment out gcs.open(... ? I don't want that commented out. I'd really love to just upload straight to cloud storage using that call, and not having to use anything related to blobs. What's the easiest way to accomplish this given that I'm stuck with using the AClassForm and the ferris framework?
class AClassForm(forms.model_form(AClass, exclude=('image_url') ) ):
image = FileField(u'Image File')
class AClasses(Controller):
class Meta:
Model = AClass
prefixes = ('admin',)
components = (scaffold.Scaffolding, Upload)
Form = AClassForm
admin_list = scaffold.list
admin_view = scaffold.view
admin_edit = scaffold.edit
admin_delete = scaffold.delete
def admin_add(self):
self.scaffold.ModelForm = AClassForm
self.scaffold.form_encoding = "multipart/form-data"
def before_save_callback(controller,container, item):
image = self.request.params["image"]
object_name = blobstore.parse_file_info(image).gs_object_name.split('/')[-1]
upload_settings = settings.get("upload")
url = upload_settings["url"]
bucket = upload_settings["bucket"]
#send to the cloud
#write a task to execute this?
item.image_url = url % (bucket, object_name)
#gcs_file= gcs.open("/".join(["", bucket, object_name]),
# 'w', content_type="image/jpeg",
#options={'x-goog-acl': 'public-read'} )
#gcs_file.write(item.image)#.file.encode('utf-8')) #
#gcs_file.close()
return
self.events.scaffold_before_save += before_save_callback
return scaffold.add(self)
I am not sure how Ferris works internally but you can use cloudstorage directly.
My image storage wrapper, provides resizing and returns a public URL to the upload for serving directly from storage.
import urlparse
from google.appengine.api import app_identity, blobstore, images
import cloudstorage
class ImageStorage(object):
def __init__(self, base_uri):
self.base_uri = "/{}/{}".format(app_identity.get_default_gcs_bucket_name(), base_uri.lstrip('/'))
def put(self, image, name, mime=None, width=None, height=None):
"""Puts an image into the Google Cloud Storage"""
if width or height:
image = images.resize(image, width=width or height, height=height or width)
mime = 'image/png' # resize defaults to output_encoding=PNG
options = {'x-goog-acl': 'public-read'}
with cloudstorage.open(self.make_key(name), 'w', content_type=mime, options=options) as fp:
fp.write(image)
return self.get_url(name)
def get_url(self, name):
"""Gets the url for an image from Google Cloud Storage"""
key = self.make_key(name)
# must be prefixed with /gs for the blob store to know it is from gcs
# https://cloud.google.com/appengine/docs/python/blobstore/#Python_Using_the_Blobstore_API_with_Google_Cloud_Storage
url = images.get_serving_url(blobstore.create_gs_key('/gs' + key))
# s/https/http/ if running under dev_appserver.py
# if not config.isLocal:
# parts = urlparse.urlparse(url)
# secure_parts = ('https',) + parts[1:]
# url = urlparse.urlunparse(secure_parts)
return url
def make_key(self, name):
"""Makes an item name key for Google Cloud Storage"""
return '%s/%s' % (self.base_uri, name)
Usage inside of a subclass of webapp2.RequestHandler. This stores the file with name "image" in the default bucket for your app in cloud storage at path /some/bucket/path/my-image-name.
thumb = self.request.POST["image"]
if hasattr(thumb, 'value'):
gs_base_uri = '/some/bucket/path'
image_storage = ImageStorage(gs_base_uri)
thumb_fn = 'my-image-name'
session_thumb_url = image_storage.put(image=thumb.value,
name=thumb_fn,
mime=thumb.type,
width=300, height=300)
return session_thumb_url
As I understand it, if you're using the Upload component of Ferris you can't escape the Blobstore, but the following comes pretty close. You don't have to use the Form class if you don't want to, I rarely use it myself. So imagine the following Controller:
from ferris import Controller, route
from ferris.components.upload import Upload
import cloudstorage as gcs
from google.appengine.ext import blobstore
import logging
class ImageManager(Controller):
class Meta:
components = (Upload,)
#route
def list(self):
#This just passes the upload URL to use in the form
self.context['upload_url'] = self.components.upload.generate_upload_url(uri=self.uri('image_manager:image_uploader_action'))
#route
def image_uploader_action(self):
# This gets all of the uploads passed in from the form
uploads = self.components.upload.get_uploads()
# This is the raw google cloud object reference. 'myfile' is the name of the upload field in the html form
file_gcs_obj_name = uploads['myfile'][0].cloud_storage.gs_object_name
# This is the blobstore key just for giggles
file_blobstore_key = uploads['myfile'][0].key()
# This will get rid of the preceeding junk you don't need i.e. "/gs/yadda/yadda"
clean_file_name = file_gcs_obj_name[3:]
# This is the name of the file as it was uploaded by the end-user
file_name_friendly = uploads['myfile'][0].filename
# This is the actual file, with this you can do whatever you want
the_actual_image = gcs.open(clean_file_name,'r')
# The file name by default is long and ugly, lets make a copy of the file with a more friendly name
new_filename = '/mydomain.appspot.com/'+file_name_friendly
gcs.copy2(clean_file_name,new_filename)
# We can generate a serving URL by using the blobstore API
valid_blob_reference = blobstore.create_gs_key('/gs'+new_filename)
file_serving_url = images.get_serving_url(valid_blob_reference)
logging.info('the serving url is: %s'% file_serving_url)
# delete the original image from cloud storage
gcs.delete(clean_file_name)
# Delete the original image from blobstore
blobstore.delete(file_blobstore_key)
# Close the file
the_actual_image.close()
return 'Done. go check the cloud storage browser'
Now all you need is the HTML form. You can use something like this:
{% extends "layouts/default.html" %}
{% block layout_content %}
<form name="myform" action="{{upload_url}}" method="POST" enctype="multipart/form-data">
<input type="file" name="myfile" id="fileToUpload">
<input type="submit" value="Upload File" name="submit">
</form>
{% endblock %}
Ferris is still going to place a file in the blobstore but you can delete it after you've used the cloudstorage.copy2() function. That function is fairly new so remember to update your cloudstorage package, you can download the latest copy from Google or Pypi (https://pypi.python.org/pypi/GoogleAppEngineCloudStorageClient/1.9.22.1)
Hope this helps.
I am working on an app to process emails hitting my mailbox. I have modified my mail settings to forward incoming mails to myapp. The mails reaching myapp will be routed to the handler script ("handle_incoming_email.py") where it will be processed. My app.yaml file looks like this
app.yaml
application: myapp
version: 1-1
runtime: python27
api_version: 1
threadsafe: false
default_expiration: "360d"
handlers:
- url: /_ah/mail/.+
script: myapp/utils/handle_incoming_email.py
The mail handler script is given below
handle_incoming_email.py
import logging
import urllib
import base64
import traceback
from google.appengine.ext import webapp
from google.appengine.ext import blobstore
from google.appengine.api import urlfetch
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext.webapp.mail_handlers import InboundMailHandler
class ReceiveEmail(InboundMailHandler):
def receive(self, message):
try:
for filename, filecontents in message.attachments:
if filecontents.encoding:
# data = filecontents
data = filecontents.decode()
# data = filecontents.payload
# data = filecontents.payload.decode()
# data = base64.b64decode(filecontents.decode())
# data = base64.b64decode(filecontents.payload)
# data = base64.b64decode(filecontents.payload.decode())
upload_url = blobstore.create_upload_url('http://myapp.appspot.com/settings/saveItem/')
form_fields = {'field_name': data}
form_data = urllib.urlencode(form_fields)
result = urlfetch.fetch(url=upload_url,
payload=form_data,
method=urlfetch.POST,
headers={'Content-Type': 'application/x-www-form-urlencoded'})
logging.info(result)
except Exception, e:
traceback.print_exc()
application = webapp.WSGIApplication([ReceiveEmail.mapping()], debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
My requirement is to create an entity corresponding to each mail with attachment. For this, I need to parse the attachment from the mail and upload it to blobstore. However, when I try to upload the attachment to blobstore, I get the following error:
The request's content type is not accepted on this URL.
As you can see in the commented code in "handle_incoming_email.py", I tried different methods (trial-and-error) to get the data correct, but to no avail.
Could someone guide me in fixing this!
Thanks!!!
I think this code sample will solve you problem. You probably need to use only encode_multipart_formdata function from this code. And do not forget to set properly content type.
class BlobstoreUpload(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file')
blob_info = upload_files[0]
return self.response.write(blob_info.key())
#classmethod
def encode_multipart_formdata(cls, fields, files, mimetype='image/png'):
"""
Args:
fields: A sequence of (name, value) elements for regular form fields.
files: A sequence of (name, filename, value) elements for data to be
uploaded as files.
Returns:
A sequence of (content_type, body) ready for urlfetch.
"""
boundary = 'paLp12Buasdasd40tcxAp97curasdaSt40bqweastfarcUNIQUE_STRING'
crlf = '\r\n'
line = []
for (key, value) in fields:
line.append('--' + boundary)
line.append('Content-Disposition: form-data; name="%s"' % key)
line.append('')
line.append(value)
for (key, filename, value) in files:
line.append('--' + boundary)
line.append('Content-Disposition: form-data; name="%s"; filename="%s"' % (key, filename))
line.append('Content-Type: %s' % mimetype)
line.append('')
line.append(value)
line.append('--%s--' % boundary)
line.append('')
body = crlf.join(line)
content_type = 'multipart/form-data; boundary=%s' % boundary
return content_type, body
class UserProfile(webapp2.RequestHandler):
def post(self):
picture = self.request.POST.get('picture')
# Write new picture to blob
content_type, body = BlobstoreUpload.encode_multipart_formdata(
[], [('file', name, image)])
response = urlfetch.fetch(
url=blobstore.create_upload_url(self.uri_for('blobstore-upload')),
payload=body,
method=urlfetch.POST,
headers={'Content-Type': content_type},
deadline=30
)
blob_key = response.content
Error does not display image with the following url
http://127.0.0.1:8080/serve/CrObzPCoJfjG4ESUUb0hjw==
Image does exist in the blobstore can be checked in admin
My route
Dope on redirect routes
RedirectRoute('/serve/[a-zA-Z0-9-_]', handlers.ServeHandler, name='ServeHandler'),
My code:
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
import urllib
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
logging.info("SERVE " + str(resource))
resource = str(urllib.unquote(resource))
blob_info = blobstore.BlobInfo.get(resource)
self.send_blob(blob_info)
class FetchHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
upload_files = self.get_uploads('file') # 'file' is file upload field in the form
blob_info = upload_files[0]
logging.info("FOUND blob info" + str(blob_info))
self.redirect('/serve/%s' % blob_info.key())
class ImageHandler(BaseHandler):
#user_required
def get(self, **kwargs):
user_session = self.user
user_session_object = self.auth.store.get_session(self.request)
upload_url = blobstore.create_upload_url('/fetch/')
user_info = models.User.get_by_id(long( self.user_id ))
user_info_object = self.auth.store.user_model.get_by_auth_token(
user_session['user_id'], user_session['token'])
try:
params = {
"upload_url": upload_url,
"user_session" : user_session,
"user_session_object" : user_session_object,
"user_info" : user_info,
"user_info_object" : user_info_object,
"userinfo_logout-url" : self.auth_config['logout_url'],
}
return self.render_template('image.html', **params)
except (AttributeError, KeyError), e:
return "Secure zone error:" + " %s." % e
I think your problem might be on this line:
self.redirect('/serve/%s' % blob_info.key())
According to the following recent changes assuming you did update appengine to the latest release:
The Blobstore service now returns the created filename instead of the blobKey when using Cloud Storage [link][1]
Have a look at the recent release notes and the changes that came with it.
I think they URL that you are providing to the create_upload_url is the wrong one, since you are defining it like /upload/.
Add the forward slash in the end and it should work:
upload_url = blobstore.create_upload_url('/upload/')
Comparing your code (form an earlier revision of your question) to some code I have working, I suspect that you might not want the trailing / on the /upload/ route (i.e., use /upload instead.
I'm not familiar with RedirectRoute, though.
I am trying to get URL of Image(blob field of GAE):
class Product(db.Model):
name = db.StringProperty()
price = db.FloatProperty()
added = db.DateTimeProperty(auto_now_add=True)
image = db.BlobProperty(default=None)
url = images.get_serving_url(movie.image)
Handler of serve image:
def result(request):
product = Product()
product.name = "halva"
url = 'http://echealthinsurance.com/wp-content/uploads/2009/11/minnesota.jpg'
product.image = db.Blob(urlfetch.Fetch(url).content)
product.put()
template = loader.get_template("result.html")
context = RequestContext(request,
{
"result" : u"Add"})
return HttpResponse(template.render(context))
But i get except:
UnicodeDecodeError:
When try to ignore this exception(that was bug in Python 2.7) I get exception in other place.
And after that i try to encode Image to 'latin-1'('utf-8' don't work):
enc_img = movie.image.decode("latin-1")
url = images.get_serving_url(enc_img)
Result: url has a view like binary file:
"ÝêÓ9>èýÑNëCf Äàr0xã³3Ï^µ7±\íQÀ¡>.....ÕÝ£°Ëÿ"I¢¶L`ù¥ºûMþÒ¸ÿ+ÿL¢ï£ÿÙ' alt="" />"
How I get url to show dynamic image in template?
You are confusing two different things here.
If you are storing your image in a db.BlobProperty (code doesn't show you are doing this, but the Schema you have is using db.BlobProperty) this means your handler has to serve the image.
However you are using image.get_serving_url, which takes a BlobKey instance which comes from storing an Image in the BlobStore https://developers.google.com/appengine/docs/python/blobstore/blobkeyclass which is a completely different thing to what you are doing.
You will need to work out what you want to do, store an image (max size 1MB) in a BlobProperty and provide a handler that can serve the image, or upload it to the BlobStore and serve it from there
images.get_serving_url takes a BlobKey. Try:
enc_img = movie.image
url = images.get_serving_url(enc_img.key())
I have successfully uploaded a file to blobstore using this code.
But I am unable to download it.
What I am doing is:
`class PartnerFileDownloadHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, blob_key):
resource = str(urllib.unquote(blob_key))
logging.info('I am here.') //This gets printed successfully.
blob_info = blobstore.BlobInfo.get(blob_key)
logging.info(blob_info) //This gets logged too.
self.send_blob(blob_info)`
I have also tried:
blobstore.BlobReader(blob_key).read()
and I get file data in string form but I can not write it to file, as local file system can not be accessed from within a handler, I guess.
The way I am uploading a file is the only way in my project so I can not use the usual way specified in the Google's official tutorial. Also The file I am uploading to blobstore is not present at my local file syatem, I pick it from a URL, perhaps this is the problem why I am not able to download the file.
Any suggestions?
Thanks
Perhaps you should use resource instead of blob_key from your code sample?
class PartnerFileDownloadHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, blob_key):
resource = str(urllib.unquote(blob_key))
self.send_blob(resource)
you can use DownloadHandler as this:
from mimetypes import guess_type
def mime_type(filename):
return guess_type(filename)[0]
class Thumbnailer(blobstore_handlers.BlobstoreDownloadHandler):
def get(self , blob_key):
if blob_key:
blob_info = blobstore.get(blob_key)
if blob_info:
save_as1 = blob_info.filename
mime_type=mime_type(blob_info.filename)
self.send_blob(blob_info,content_type=mime_type,save_as=save_as1)