How to manipulate files in google app engine datastore - google-app-engine

Question

My problem revolves around a user making a text file upload to my app. I need to get this file and process it with my app before saving it to the datastore. From the little I have read, I understand that user uploads go directly to the datastore as blobs, which is ok if I could then get that file, perform operations on it(meaning change data inside) and then re-write it back to the datastore. All these operations need to be done by the app.
Unfortunately from the datastore documenation, http://code.google.com/appengine/docs/python/blobstore/overview.html
an app cannot directly create a blob in the datastore. That's my main headache. I simply need a way of creating a new blob/file in the datastore from my app without any user upload interaction.

Answer 1

blobstore != datastore.
You can read and write data to the datastore as much as you like so long as your data is <1MB using a db.BlobProperty on your entity.
As Wooble comments, the new File API lets you write to the blobstore, but unless you are incrementally writting to the blobstore-file using tasks or something like the mapreduce library you are still limited by the 1MB API call limit for reading/writing.

Answer 2

Thanks for your help. After many sleepless nights, 3 App Engine Books and A LOT of Googling, I've found the answer. Here is the code (it should be pretty self explanatory):
from __future__ import with_statement
from google.appengine.api import files
from google.appengine.ext import blobstore
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write('Hello WOrld')
form=''' <form action="/" method="POST" enctype="multipart/form-data">
Upload File:<input type="file" name="file"><br/>
<input type="submit"></form>'''
self.response.out.write(form)
blob_key="w0MC_7MnZ6DyZFvGjgdgrg=="
blob_info=blobstore.BlobInfo.get(blob_key)
start=0
end=blobstore.MAX_BLOB_FETCH_SIZE-1
read_content=blobstore.fetch_data(blob_key, start, end)
self.response.out.write(read_content)
def post(self):
self.response.out.write('Posting...')
content=self.request.get('file')
#self.response.out.write(content)
#print content
file_name=files.blobstore.create(mime_type='application/octet-stream')
with files.open(file_name, 'a') as f:
f.write(content)
files.finalize(file_name)
blob_key=files.blobstore.get_blob_key(file_name)
print "Blob Key="
print blob_key
def main():
application=webapp.WSGIApplication([('/', MainHandler)],debug=True)
util.run_wsgi_app(application)
if __name__=='__main__':
main()