First I put image to storage:
import cloudstorage as gcs
...
path = '/bucket/folder/image.jpg'
with gcs.open(path, 'w') as f:
f.write(data)
Then I get serving url:
url = images.get_serving_url(None, filename='/gs{}'.format(self.path),
secure_url=True)
Serving url generally works as expected, the thing is I'm not using blob_key, only filename (path in storage).
I wonder how to delete serving_url now, since sdk method only accepts blob_key
def delete_serving_url(blob_key, rpc=None):
"""Delete a serving url that was created for a blob_key using get_serving_url.
Args:
blob_key: BlobKey, BlobInfo, str, or unicode representation of BlobKey of
blob that has an existing URL to delete.
rpc: Optional UserRPC object.
Raises:
BlobKeyRequiredError: when no blobkey was specified.
InvalidBlobKeyError: the blob_key supplied was invalid.
Error: There was a generic error deleting the serving url.
"""
https://cloud.google.com/appengine/docs/python/refdocs/google.appengine.api.images#google.appengine.api.images.delete_serving_url
The Using the Blobstore API with Google Cloud Storage example shows how to obtain an equivalent blob_key for GCS:
blob_key = CreateFile(main.BUCKET + '/blobstore_serving_demo')
From that link:
Note: Once you obtain a blobKey for the Google Cloud Storage object, you can pass it around, serialize it, and otherwise use it
interchangeably anywhere you can use a blobKey for objects stored
in Blobstore. This allows for usage where an app stores some data in
blobstore and some in Google Cloud Storage, but treats the data
otherwise identically by the rest of the app. (However, BlobInfo
objects are not available for Google Cloud Storage objects.)
So you should be able to generate a blobKey for your file and call get_serving_url and delete_serving_url with it.
You could also use GCS object premissions to prevent access to the file, see Setting object permissions and metadata.
Related
I'm currently using a dropbox client js script to push zip files to a folder (in test, a couple of k, in production, a couple of hundred meg) - there currently isn't a server/back end, so am posting from an arraybuffer, not a server url.
var zip = new JSZip();
zip.file("test.txt", "Hello World\n");
var content = zip.generate({type:"arraybuffer"});
// ... code to pick a dropbox folder ...//
client.writeFile(url+"/"+fileName, content, function(error){ ... etc
This all works fine - client is able to write the binary file (which Dropbox's own Saver is unfortunately unable to do). I'm trying to see if Kloudless is able to perform the same, since I also need to support google, box, etc at some point. https://github.com/kloudless/file-explorer/'s documentation about its saver says the files are an array of urls ..
explorer({
...
files: [{
"url": "http://<your image url>",
"name": "filename.extension"
},
It doesn't seem to like local storage file references using URL.createObjectURL(blob), so I'm guessing the api is telling the remote services to pull the files rather than pushing their data.
You are correct that the Kloudless API backend servers stream the file from the URL to the final destination in whichever cloud service you would like the file to be uploaded to (e.g. a folder in a Dropbox account).
If the files are present only on the client-side, I would recommend using the Kloudless Chooser to instead prompt the user to choose a folder to save the files in, and then handle uploading the file data to that destination via the client-side manually.
To do this, refer to this example configuration: https://jsfiddle.net/PB565/139/embedded/
I have set retrieve_tokens to true so that my client-side JavaScript will receive not just the metadata of the folder the user chooses to upload the data to but also the Bearer token to use to gain access to the user's account. This allows the client-side JavaScript to then make upload or multipart upload requests to Kloudless to upload the file data to that folder. The advantage of multipart uploads is that an error uploading one chunk wouldn't require the whole upload to be retried.
Be sure to add the domain you are hosting the File Explorer on to your Kloudless App's Trusted Domains (on the App Details page) so that it can in fact receive the Bearer token in the response JS callback. In my JSFiddle example, I would have to add 'fiddle.jshell.net' to my app's list of Trusted Domains to be able to receive the Bearer token to perform further requests from the client side to the Kloudless API.
How do you retrieve a blobkey from the datastore in google appengine using java.?
Query= new Query("__BlobInfo__"); // this doesnt work
BlobKey blobKey = new BlobKey();
// Is there a Query like feature as in the datastore to access entities for blobstore... i need the key of the blob here to view the image on a seperate jsp page//
blobstoreService .serve(blobKey ,res);
I have read Open a file from urlfetch in GAE and Google App Engine, How to make native file object? but still can't figure out how to save a .pdf file from an URL to the Google datastore or blobstore.
Any hint or a pointer to a working example?
I'm using Python and my code is essentially the one from https://developers.google.com/appengine/docs/python/urlfetch/?hl=it-IT&csw=1
from google.appengine.api import urlfetch
url = "http://www.gutenberg.org/files/1342/1342-pdf.pdf"
result = urlfetch.fetch(url)
if result.status_code == 200:
doSomethingWithResult(result.content)
something like
First your model to store the PDF in
class PDFStore(ndb.Model):
content = ndb.BlobProperty()
The create an instance of that model, put the data into it the save it.
def doSomethingWithResult(result):
PDF = result.content
blobstore_data = PDFStore()
blobstore_data.content = PDF
blobstore_data.put()
It's exactly as in the first link you put up: Open a file from urlfetch in GAE
I am storing a file in Google App Engine using Google Cloud Storage. The file is loaded fine but the serve returns a file interpreted as a binary not as the original mime type. You'll find hereafter the code. Has anyone an idea of what is happening ?
I. Store file
GSFileOptionsBuilder optionsBuilder = new GSFileOptionsBuilder()
.setBucket(BUCKET_NAME)
.setKey(objectId)
.setAcl("project-private")
.setMimeType(mimeType)
.setContentDisposition("attachment;filename "+item.getName());
II. Serve file
blobKey = blobstoreService.createGsBlobKey("/gs/"+BUCKET_NAME+"/"+ fileName);
blobstoreService.serve(blobKey, resp);
III. Difference with serving a file in the blobstore
I had a piece of code which used to work. The problem is that apparently the BlobInfo object is only for objects stored in the blobstore and not in the Google Cloud Storage
BlobInfo blobInfo = blobInfoFactory.loadBlobInfo(blobKey);
resp.setContentLength(new Long(blobInfo.getSize()).intValue());
resp.setHeader("content-type", blobInfo.getContentType());
resp.setHeader("content-disposition", "attachment; filename=" +
blobInfo.getFilename());
blobstoreService.serve(blobKey, resp);
Any help is very welcome !
Thanks,
Hugues
Because you're serving the blob yourself, using the blob serving service, it's up to you to set all the HTTP headers, including the content type, correctly. If you want to use the content type of the stored object, you should fetch it and set it in the headers yourself.
Alternatively, you could link directly to the object's path in Google Storage, in which case it will be served by the Google Storage infrastructure, with the correct mimetype.
I'm trying to store audio files in google app engine's blobstore and play them in a browser. The problem I'm running into is that the data I'm getting in the browser is the actual mp3 data. I was expecting to get a url to play the mp3 in the blobstore. So, my question is, what do I need to change to get a url to play the blob instead of the audio data?
Here is my server side handler.
class ServeBlobHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self):
user = users.get_current_user()
query = db.GqlQuery("SELECT * FROM AudioData Where userId = :1", user.user_id())
results = query.fetch(limit=300)
for dStoreEntry in results:
entityBlobInfo = dStoreEntry.audioBlob
self.send_blob(entityBlobInfo)
This is the client side.
$.ajax({
url : '/serve_blob/audio/',
type : 'GET',
dataType : 'text',
success : function(data) {
alert('GET, audio data : \n '+ data );
}
});
The URL of the page that you're currently fetching the data from is the URL of the MP3. You'll need to use a web-based player of some sort to play it.
What Content-type header does your browser get for mp3 request? I'm guessing it's application/octet-stream
See what Blobstore docs say about upload:
If you don't specify a content type, the Blobstore will try to infer it from the
file extension. If no content type can be determined, the newly created blob is
assigned content type application/octet-stream
Go to GAE admin pages and check Blob Viewer to see under what content type was assigned to your mp3 files.
Get JPlayer - http://www.jplayer.org/
And then your example should work fine. We use it with appengine blobstore in java and it's great. The url from the blobstore will work in jplayer.
You can also set cache headers on your blob urls if you rewrite them to remove any query parameters and save yourself the costs of serving each stream.