Is there a consistent code-base that allows me to upload a zip file to both GAE and Tomcat-based servers, extract the contents (plain-text files), and process them?
Both supports Java, so you can just use Java :)
In all seriousness, processing file uploads can be done with Apache Commons FileUpload and extracting them can be done with java.util.zip API.
See also the answers on those similar questions which are asked last two days, probably by your classmates/friends:
JSP/Servlets: How do I Upload a zip file, unzip it and extract the CSV file…
Upload a zip file, unzip and read file
Related
Working on porting GAE Datastore files to a new data base and need a fully readable, idealy csv format.
I am not getting the most readable information when i pulled my google datastore files into atom editor. Is there a better way to read google datastore files or is there an atom plugin
I'm feeling stupid, but I want to know how GitHub and Dropbox store user files, because I have a similar problem and I need to store user's project files .
Is it just like storing project files somewhere in the server and refer to the location as a field in the database, or there are other better methods ?
Thanks.
GitHub uses Git to store repositories, and accesses those repos from their Ruby application. They used to do this with Grit, a Ruby library. Grit was written to implement Git in Ruby but has been replaced with rugged. There are Git reimplementations in other languages like JGit for Java and Dulwich for Python. This presentation gives some details about how GitHub has changed over the years and is worth watching/browsing the slides.
If you wanted to store Git repositories, what you'd want to do is store them on a filesystem (or a cluster thereof) and then have a pointer in your database to point to where the filesystem is located, then use a library like Rugged or JGit or Dulwich to read stuff from the Git repository.
Dropbox stores files on Amazon's S3 service and then implements some wrappers around that for security and so on. This paper describes the protocol that Dropbox uses.
The actual question you've asked is how do you store user files. The simple answer is... on the filesystem. There are plugins for a lot of popular web frameworks for doing user file uploads and file management. Django has Django-Filer for instance. The difficulty you'll encounter in rolling your own file upload management system is building a sensible way to do permissions (so users can only download the files they are entitled to download), so it is worth looking into how the various framework plugins do it.
How can i include txt files and access to them within play framework? I need to load some text from .txt files depending on user request. I'm used to access files from inside jars and thinking in deploying the web as a runnable jar.
You can do exactly the same thing in Play. It doesn't matter whether your application is packaged as a jar file or not.
The resource file just needs to be on the CLASSPATH.
I have a GAE/Python application that is an admin program that allows people all over the world to translate template files for a large GAE application into their local language (we cannot use auto translation because many idioms and the like are involved). The template files have been tokenized and text snippets in the various languages are stored in a GAE datastore (there are thousands of template files involved).
I therefore need to be able to write files to a folder.
I have the following code:
with open("my_new_file.html", "wb") as fh:
fh.write(output)
I have read that GAE blocks the writing of files. If that is true, is there some way to get around it?
If I cannot write the file direct, does anyone have a suggestion for how I accomplish the same thing (e.g. do something with a web-service that does some kind of round trip to download and then upload the file)?
I am a newbie to GAE/Python, so please be specific.
Thanks for any suggestions.
you could use google app engine blobstore or BlobProperty in datastore to store blobs/files
for using blobstore (up to 2GB)
https://developers.google.com/appengine/docs/python/blobstore/
for using datastore blobs (only up to 1MB)
https://developers.google.com/appengine/docs/python/datastore/typesandpropertyclasses#Blob
Filesystem is read only in many cloud system and GAE is too. In a virtual world, where the OS and machine are virtual, the filesystem is least reliable place to store anything
I would suggest using any of BLOB, Google Cloud Storage, Google Drive or even go a setp further and store in any external provider like Amazon S3 etc.
Use the files API:
https://developers.google.com/appengine/docs/python/googlestorage/overview
Adding some extra code you can use it like the normal Python file API:
with files.open(writable_file_name, 'a') as f:
f.write('Hello World!')
While this particular link describes it in relation with Google Cloud Storage (GCS) you can easily replace the GCS-specific pieces and use blobstore as a storage backend.
The code can be found here:
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/files/
I want to sending multiple files to solr using curl.How i can do it ?
I can done with only one file with command for example:
curl
"http://localhost:8983/solr/update/extract?literal.id=paas2&commit=true"
-F "file=#cloud.pdf"
Anyone can help me,
Tks
The api does not support passing multiple files for extraction.
Usually the last file will be the only one thats gets uploaded and added.
You can have individual files indexed as separate entities in Solr.
OR One way to upload multiple files is to zip these files and upload the zip file.
There is one issue with Solr indexing zip files and you can try the SOLR-2332 Patch
i using apache solr 4.0 Beta which have capability to upload multiple file and generate id for each file uploaded using post.jar and It's very helpfull for me.
Let'see on :
http://wiki.apache.org/solr/ExtractingRequestHandler#SimplePostTool_.28post.jar.29
Thanks all :)
my problem have solved :)