Just this, I'd like to stream uploaded filed directly from the net to the filesystem to avoid out of memory errors. Can I do it with CakeRequest::input()? Is there any other way?
Maybe it's best to read the API documentation CakeRequest::input() or the source;
http://api.cakephp.org/2.3/source-class-CakeRequest.html#876
According to the source, 'input()' reads directly from php://input via the _readInput() method:
However, if I read that part of the source code correctly, it will read the entire stream in memory before returning its content. So I don't think this will give you what you want.
There may be other solutions, maybe a plugin exists. However, you may write your own implementation, using the CakeRequest as an example?
You may also check the HttpSocket class
Related
I want to play a media file from a memory stream using LibVLC like so:
//Ideally it would go like this:
LibVLC.MediaFromStream = new MemoryStream(File.ReadAllBytes(File_Path));
Of course this is a very oversimplified version of what I want but hopefully it conveys what I am looking for.
The reason being that I want there to be a good amount of portability for what I'm doing without having to track file locations and such. I'd rather have a massive clump of data in a single file that can be read from than have to track the locations of one or many more files.
I know this has something to do with the LibVLC IMEM Access module. However, looking at what information I've been able to find on that, I feel like I've been tossed from a plane and have just a few minutes to learn how to fly before I hit the ground.
See my answer to a similar question here:
https://stackoverflow.com/a/31316867/2202445
In summary, the API:
libvlc_media_t* libvlc_media_new_callbacks (libvlc_instance_t * instance,
libvlc_media_open_cb open_cb,
libvlc_media_read_cb read_cb,
libvlc_media_seek_cb seek_cb,
libvlc_media_close_cb close_cb,
void * opaque)
allows just this. The four callbacks must be implemented, although the documentation states the seek callback is not always necessary, see the libVlc documentation. I give an example of a partial implementation in the above answer.
There is no LibVLC API for imem, at least not presently.
You can however still use imem in your LibVLC application, but it's not straightforward...
If you do vlc -H | grep imem you will see something like this (this is just some of the options, there are others too):
--imem-get <string> Get function
--imem-release <string> Release function
--imem-cookie <string> Callback cookie string
--imem-data <string> Callback data
You can pass values for these switches either when you create your libvlc instance via libvlc_new(), or when you prepare media via libvlc_media_add_option().
Getting the needed values for these switches is a bit trickier, since you need to pass the actual in-memory address (pointer) to the callback functions you declare in your own application. You end up passing something like "--imem-get 812911313", for example.
There are downsides to doing it this way, e.g. you may not be able to seek backwards/forwards in the stream.
I've done this successfully in Java, but not C# (never tried).
An alternative to consider if you want to play the media data stored in a file, is to store your media in a zip or rar since vlc has plugins to play media from directly inside such archives.
I have an issue whilst streaming to a file, I'm sure there is a simple solution but I'm struggling to find it! What I'm attempting to do is very straightforward, I'm getting the contents of a class and serializing them into XML and streaming them to a file. The code I'm using is:
ObservableCollection<RackItemViewModel> rackVMs = ProjectTree.GetTreeData();
XmlSerializer serializer = new XmlSerializer(typeof(RackItem));
using (TextWriter tw = new StreamWriter(filename, false))
{
foreach (RackItemViewModel VM in rackVMs)
serializer.Serialize(tw, VM.RackItem);
}
ProjectTree.GetTreeData() just returns the data to be serialized. If I run the program and save the data it all works as expected, the data is saved and can be read back with Deserialize. The problem I'm having is when I perform more than one save. If I save one set of data to one file and then another set of data to another file, the first file is correct but the second file is a concatenation of the first file and the second! It seems that either the stream or the XMLSerializer are not releasing their contents between saves. I've tried using writefile instead of Stream and I still get the same issue, I've also tried flushing and closing the stream but this has no effect. Needless to say if I close and restart the application between saves it all works fine.
Could anyone tell me what I'm doing wrong please?
Before writing to new file try flushing the stream tw.Flush().
I thought I'd tidy up this thread as I've managed to solve the problem. It turns out that it was nothing to do with the serializing or streaming of the data. The data buffer being written wasn't fully releasing the data between writes. I was checking the View Model object which was OK but the object being written (RackItem), wasn't following suite. Silly error on my part. Thanks for the suggestions.
I made a wrapper for some basic things in libxml2, stuff like grabbing element content, stepping into children nodes etc.
My super has just asked me to make sure I'm parsing the XML file serially and not loading the entire DOM into memory.
I'm pretty sure the I'm doing it serially, but I couldn't find any documentation on parsing one way or the other.
Any help is appreciated, thanks!
libxml2 can operate in either mode. It just depends how your code uses it. You can either parse the full file into a DOM, or use Sax callbacks to parse serially. What does your parsing code look like?
There are two different APIs you can use.
xmlTextReader is a streaming reader that you'd want to use, calling xmlTextReaderRead() repleatedly to advance the parser through the file.
http://xmlsoft.org/xmlreader.html
If you're working with xmlDocPtr/xmlNodePtr objects returned by things like xmlParseFile, then that's the tree-based DOM API.
http://xmlsoft.org/examples/index.html#tree1.c
I found myself passing InputStream/OutputStream objects around my application modules.
I'm wondering if it's better to - save the content to disk and pass something like a Resource between the various methods calls - use a byte[] array instead of having to deal with streams everytime.
What's your approach in these situations?Thanks
Edit:
I've a Controller that receives a file uploaded by the user. I've an utility module that provides some functionality to render a file.
utilityMethod(InputStream is, OutputStream os)
The file in InputStream is the one uploaded by the user. os is the stream associated with the response. I'm wondering if it's better to have the utility method to save the generated file in a .tmp file and return the file path, or a byte[], etc. and have the controller to deal with the outputStream directly.
I try to keep as much in RAM as possible (mostly because of performance reasons and RAM is cheap). So I'm using a FileBackedBuffer to "save" data of unknown size. It has a limit. When less than limit bytes are written to it, it will keep them in an internal buffer. If more data is written, I'll create the actual file. This class has methods to get an InputStream and an OutputStream from it, so the using code isn't bothered with the petty details.
The answer actually depends on the context of the problem, which we dont know.
So, imagining the most generic case, I would create two abstractions. The first abstraction would take InputStream/OutputStream as parameters, whereas the other would take byte[].
The one that takes streams can read and pass the data to the byte[] implementation. So now your users can use both the stream abstraction and byte[] abstraction based on thier needs/comfort.
I want to be able to read the content of pdf files. I need to do that with C on Linux.
The closer i can get to this was here but I think Haru can only create pdf and is not able to read them (not 100% sure).
PS: I only need the plain text from pdf
Check out libpoppler. I've never used it work extracting text, just querying PDF attributes. It's pretty easy to use.
How well do you need to parse them?
Just extracting strings should be relatively easy, fully accurate rendering is harder.
Take a look at the source for evince or ghostscript?
This is for C++ but might be a good starting point for understanding PDF structure http://www.codeproject.com/KB/cpp/ExtractPDFText.aspx (sorry wrong link before)
Another possible, though I've never used it is VersyPDF. It claims to allow you to edit PDFs ... http://versypdf.sybrex-systems-ltd.qarchive.org/