I have successfully uploaded a file to blobstore using this code.
But I am unable to download it.
What I am doing is:
`class PartnerFileDownloadHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, blob_key):
resource = str(urllib.unquote(blob_key))
logging.info('I am here.') //This gets printed successfully.
blob_info = blobstore.BlobInfo.get(blob_key)
logging.info(blob_info) //This gets logged too.
self.send_blob(blob_info)`
I have also tried:
blobstore.BlobReader(blob_key).read()
and I get file data in string form but I can not write it to file, as local file system can not be accessed from within a handler, I guess.
The way I am uploading a file is the only way in my project so I can not use the usual way specified in the Google's official tutorial. Also The file I am uploading to blobstore is not present at my local file syatem, I pick it from a URL, perhaps this is the problem why I am not able to download the file.
Any suggestions?
Thanks
Perhaps you should use resource instead of blob_key from your code sample?
class PartnerFileDownloadHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, blob_key):
resource = str(urllib.unquote(blob_key))
self.send_blob(resource)
you can use DownloadHandler as this:
from mimetypes import guess_type
def mime_type(filename):
return guess_type(filename)[0]
class Thumbnailer(blobstore_handlers.BlobstoreDownloadHandler):
def get(self , blob_key):
if blob_key:
blob_info = blobstore.get(blob_key)
if blob_info:
save_as1 = blob_info.filename
mime_type=mime_type(blob_info.filename)
self.send_blob(blob_info,content_type=mime_type,save_as=save_as1)
Related
I am working on a REST python application and I have picked flask_restless to build endpoints connected to the database. One of the tables I would like to manage is storing binary files as blobs (LargeBinary).
I have noticed, though, that flask_restless requires json data for POST requests. I tried to apply base64 to the binary file contents and wrap it with json, but ultimately flask_restless passed file contents to sqlalchemy as a string and the SQLite backend complained that it requires bytes input (quite rightly so).
I tried searching the interwebs for a solution, but either I am formulating my query incorrectly, or actually there is none.
So, is there a way to configure the endpoint managed with flask_restless to accept binary file as an attachment? Or rather the suggested solution would be to setup the endpoint for that particular table directly with flask (I did that before in another app), away from flask_restless?
It turns out that sending an attachment is not possible.
So I dug deeper into how to send base64-encoded attachments which would then be saved as blobs.
For that I used pre- and post-processing facility of flask_restless:
def pp_get_single_image(result=None, **kw):
import base64
result['image'] = base64.b64encode(result['image']).decode('utf8')
def pp_get_many_images(result=None, search_params=None, **kw):
result['objects'] = [pp_get_single_image(d) or d for d in result['objects']]
def pp_post_image_in(data=None, **kw):
import base64
data['image'] = base64.b64decode(data['image'])
def pp_post_image_out(result=None, **kw):
import base64
result['image'] = base64.b64encode(result['image']).decode('utf8')
postprocessors=dict(GET_SINGLE=[pp_get_single_image], GET_MANY=[pp_get_many_images], POST=[pp_post_image_out])
preprocessors=dict(POST=[pp_post_image_in])
manager = flask_restless.APIManager(app, flask_sqlalchemy_db=db)
manager.create_api(Image, methods=['GET', 'POST', 'DELETE'],
postprocessors=pp_image.postprocessors,
preprocessors=pp_image.preprocessors)
Iam trying to upload files to s3 using Boto3 and make that uploaded file public and return it as a url.
class UtilResource(BaseZMPResource):
class Meta(BaseZMPResource.Meta):
queryset = Configuration.objects.none()
resource_name = 'util_resource'
allowed_methods = ['get']
def post_list(self, request, **kwargs):
fileToUpload = request.FILES
# write code to upload to amazone s3
# see: https://boto3.readthedocs.org/en/latest/reference/services/s3.html
self.session = Session(aws_access_key_id=settings.AWS_KEY_ID,
aws_secret_access_key=settings.AWS_ACCESS_KEY,
region_name=settings.AWS_REGION)
client = self.session.client('s3')
client.upload_file('zango-static','fileToUpload')
url = "some/test/url"
return self.create_response(request, {
'url': url // return's public url of uploaded file
})
I searched whole documentation I couldn't find any links which describes how to do this can someone explain or provide any resource where I can find the soultion?
I'm in the same situation.
Not able to find anything in the Boto3 docs beyond generate_presigned_url which is not what I need in my case since I have public readable S3 Objects.
The best I came up with is:
bucket_location = boto3.client('s3').get_bucket_location(Bucket=s3_bucket_name)
object_url = "https://s3-{0}.amazonaws.com/{1}/{2}".format(
bucket_location['LocationConstraint'],
s3_bucket_name,
key_name)
You might try posting on the boto3 github issues list for a better solution.
I had the same issue.
Assuming you know the bucket name where you want to store your data, you can then use the following:
import boto3
from boto3.s3.transfer import S3Transfer
credentials = {
'aws_access_key_id': aws_access_key_id,
'aws_secret_access_key': aws_secret_access_key
}
client = boto3.client('s3', 'us-west-2', **credentials)
transfer = S3Transfer(client)
transfer.upload_file('/tmp/myfile', bucket, key,
extra_args={'ACL': 'public-read'})
file_url = '%s/%s/%s' % (client.meta.endpoint_url, bucket, key)
The best solution I found is still to use the generate_presigned_url, just that the Client.Config.signature_version needs to be set to botocore.UNSIGNED.
The following returns the public link without the signing stuff.
config = Config(signature_version=botocore.UNSIGNED)
config.signature_version = botocore.UNSIGNED
boto3.client('s3', config=config).generate_presigned_url('get_object', ExpiresIn=0, Params={'Bucket': bucket, 'Key': key})
The relevant discussions on the boto3 repository are:
https://github.com/boto/boto3/issues/110
https://github.com/boto/boto3/issues/169
https://github.com/boto/boto3/issues/1415
Somebody who wants to build up a direct URL for the public accessible object to avoid using generate_presigned_url for some reason.
Please build URL with urllib.parse.quote_plus considering whitespace and special character issue.
My object key: 2018-11-26 16:34:48.351890+09:00.jpg
please note whitespace and ':'
S3 public link in aws console: https://s3.my_region.amazonaws.com/my_bucket_name/2018-11-26+16%3A34%3A48.351890%2B09%3A00.jpg
Below code was OK for me
import boto3
s3_client = boto3.client
bucket_location = s3_client.get_bucket_location(Bucket='my_bucket_name')
url = "https://s3.{0}.amazonaws.com/{1}/{2}".format(bucket_location['LocationConstraint'], 'my_bucket_name', quote_plus('2018-11-26 16:34:48.351890+09:00.jpg')
print(url)
Going through the existing answers and their comments, I did the following and works well for special cases of file names like having whitespaces, having special characters (ASCII), corner cases. E.g. file names of the form: "key=value.txt"
import boto3
import botocore
config = botocore.client.Config(signature_version=botocore.UNSIGNED)
object_url = boto3.client('s3', config=config).generate_presigned_url('get_object', ExpiresIn=0, Params={'Bucket': s3_bucket_name, 'Key': key_name})
print(object_url)
For Django, if you use Django storages with boto3 the code below does exactly what you want:
default_storage.url(name=f.name)
I used an f-string for the same
import boto3
#s3_client = boto3.session.Session(profile_name='sssss').client('s3')
s3_client=boto3.client('s3')
s3_bucket_name = 'xxxxx'
s3_website_URL= f"http://{s3_bucket_name}.s3-website.{s3_client.get_bucket_location(Bucket=s3_bucket_name)['LocationConstraint']}.amazonaws.com"
I am saving a user's image as a BlobProperty by doing:
user.image = urlfetch.fetch(image_url).content
Then I'm rendering that image using a url such as:
/image/user_id
The image must be saving because because when I do len(user.image) I get a number in the thousands. And on the local instance the image renders ok. On the deployed app, I get the following error, and when I go to the image url nothing shows in the browser:
Traceback (most recent call last):
File "/base/python27_runtime/python27_dist/lib/python2.7/wsgiref/handlers.py", line 86, in run
self.finish_response()
File "/base/python27_runtime/python27_dist/lib/python2.7/wsgiref/handlers.py", line 127, in finish_response
self.write(data)
File "/base/python27_runtime/python27_dist/lib/python2.7/wsgiref/handlers.py", line 202, in write
assert type(data) is StringType,"write() argument must be string"
AssertionError: write() argument must be string
Also, here's the handler that serves the image:
class ImageHandler(webapp2.RequestHandler):
""" Returns image based on id. """
def get(self, *args, **kwargs):
user = db.get(
db.Key.from_path('User', models.User.get_key_name(kwargs.get('id'))))
if user.image:
self.response.headers['Content-Type'] = "image/jpeg"
self.response.out.write(user.image)
else:
self.response.out.write("No image")
Just to clarify I tried both setting content-type to jpeg and png. And things are working ok on the local server. Any help would be appreciated. Thanks!
Why not write the image to the blobstore and then use the send_blob() mechanism?
http://code.google.com/appengine/docs/python/blobstore/overview.html#Serving_a_Blob
Answering my question, that fixes everything:
self.response.out.write(str(user.image))
It's confusing because the example in the docs does not cast the BlobProperty as string.
In my Grails application I need to create a file in current system in which I need to save information fetched from table in database. How to do this from within controller action? I don't have any idea of it.
I have created file as
File file=new File("file name.txt")
file.createNewFile();
then I have wrote values of MySQL database table fields in it as:
file<<patient.id
file<<patient.name
.
.
.
it stores data like continuous text but I want to have a .doc file in which data should get stored in table. I found Apache's POI for creating doc file but I am not getting how it works and how I should use it.
Not sure exactly what you want to store in a file but below is an example of how to easly write a String to a file using Apache-commons-io Which should be included in grails
import org.apache.commons.io.FileUtils;
class SomeController{
def writeToFile = {
def data = getSomeStringData();
def fileStore = new File("./path/to/files/ControllerOutput_${new Date()}.txt");
fileStore.createNewFile();
FileUtils.writeStringToFile(fileStore, data);
println("your file was created # {fileStore.absolutePath} and is ${fileStore.length()} bytes");
}
}
Does this help? If not, you need to explain exactly what your looking for.
This is a comment to Michael's answer (unfortunately I still don't have the reputation to reply on answers).
If you're struggling around the problem how to specifiy the relative path from within your controller's context, this might help you:
So if you have following folder you want to read/write files from/into"..
/myproject/web-app/temp/
you can access the file like this:
import org.codehaus.groovy.grails.commons.ApplicationHolder as AH
// getResource references to the web-app folder as root folder
Resource resource = AH.getApplication().getParentContext().getResource("/temp/myfile.txt)
I'd like to use GAE to allow a few users to upload files and later retrieve them. Files will be relatively small (a few hundred KB), so just storing stuff as a blob should work. I haven't been able to find any examples of something like this. There are a few image uploading examples out there but I'd like to be able to store word documents, pdfs, tiffs, etc. Any ideas/pointers/links? Thanks!
The same logic used for image uploads apply for other archive types. To make the file downloadable, you add a Content-Disposition header so the user is prompted to download it. A webapp simple example:
class DownloadHandler(webapp.RequestHandler):
def get(self, file_id):
# Files is a model.
f = Files.get_by_id(file_id)
if not f:
return self.error(404)
# Set headers to prompt for download.
headers = self.response.headers
headers['Content-Type'] = f.content_type or 'application/octet-stream'
headers['Content-Disposition'] = 'attachment; filename="%s"' % f.filename
# Add the file contents to the response.
self.response.out.write(f.contents)
(untested code, but you get the idea :)
It sounds like you want to use the Blobstore API.
You don't mention if you are using Python or Java so here are links to both.
I use blobstore API that admits any file upload/download up to 50 MB.