I found myself passing InputStream/OutputStream objects around my application modules.
I'm wondering if it's better to - save the content to disk and pass something like a Resource between the various methods calls - use a byte[] array instead of having to deal with streams everytime.
What's your approach in these situations?Thanks
Edit:
I've a Controller that receives a file uploaded by the user. I've an utility module that provides some functionality to render a file.
utilityMethod(InputStream is, OutputStream os)
The file in InputStream is the one uploaded by the user. os is the stream associated with the response. I'm wondering if it's better to have the utility method to save the generated file in a .tmp file and return the file path, or a byte[], etc. and have the controller to deal with the outputStream directly.
I try to keep as much in RAM as possible (mostly because of performance reasons and RAM is cheap). So I'm using a FileBackedBuffer to "save" data of unknown size. It has a limit. When less than limit bytes are written to it, it will keep them in an internal buffer. If more data is written, I'll create the actual file. This class has methods to get an InputStream and an OutputStream from it, so the using code isn't bothered with the petty details.
The answer actually depends on the context of the problem, which we dont know.
So, imagining the most generic case, I would create two abstractions. The first abstraction would take InputStream/OutputStream as parameters, whereas the other would take byte[].
The one that takes streams can read and pass the data to the byte[] implementation. So now your users can use both the stream abstraction and byte[] abstraction based on thier needs/comfort.
Related
In my mobile app I have an object implementing PropertyBusinessObject which contains numerous other objects also implementing this interface. This object structure is populated by JSON data I am getting back from my server. When I try to write this object to Storage with writeObject() I get the above error. The stacktrace shows it originating in the com.codename1.io.Util.writeObject() method where it is writing UTF-8 (limited to 64k). The developer guide does not reference any potential issues with Storage and recommends it over FileSystemStorage. My question is, is there a workaround/update for this? Would I have to revert to writing out the object structure to the filesystem? Thanks.
If you have a ridiculously long string e.g. to represent the contents of the file I would suggest rethinking that. Strings are inefficient in Codename One since we need to copy their representation into the iOS native layer. Also writing them to UTF is very wasteful if what you need is a binary representation. I suggest using a byte array.
Serializing to storage is a simple approach. It works great for small objects. If you have larger objects you might want to store them individually. You can also serialize to/from JSON so your storage data is readable.
I want to play a media file from a memory stream using LibVLC like so:
//Ideally it would go like this:
LibVLC.MediaFromStream = new MemoryStream(File.ReadAllBytes(File_Path));
Of course this is a very oversimplified version of what I want but hopefully it conveys what I am looking for.
The reason being that I want there to be a good amount of portability for what I'm doing without having to track file locations and such. I'd rather have a massive clump of data in a single file that can be read from than have to track the locations of one or many more files.
I know this has something to do with the LibVLC IMEM Access module. However, looking at what information I've been able to find on that, I feel like I've been tossed from a plane and have just a few minutes to learn how to fly before I hit the ground.
See my answer to a similar question here:
https://stackoverflow.com/a/31316867/2202445
In summary, the API:
libvlc_media_t* libvlc_media_new_callbacks (libvlc_instance_t * instance,
libvlc_media_open_cb open_cb,
libvlc_media_read_cb read_cb,
libvlc_media_seek_cb seek_cb,
libvlc_media_close_cb close_cb,
void * opaque)
allows just this. The four callbacks must be implemented, although the documentation states the seek callback is not always necessary, see the libVlc documentation. I give an example of a partial implementation in the above answer.
There is no LibVLC API for imem, at least not presently.
You can however still use imem in your LibVLC application, but it's not straightforward...
If you do vlc -H | grep imem you will see something like this (this is just some of the options, there are others too):
--imem-get <string> Get function
--imem-release <string> Release function
--imem-cookie <string> Callback cookie string
--imem-data <string> Callback data
You can pass values for these switches either when you create your libvlc instance via libvlc_new(), or when you prepare media via libvlc_media_add_option().
Getting the needed values for these switches is a bit trickier, since you need to pass the actual in-memory address (pointer) to the callback functions you declare in your own application. You end up passing something like "--imem-get 812911313", for example.
There are downsides to doing it this way, e.g. you may not be able to seek backwards/forwards in the stream.
I've done this successfully in Java, but not C# (never tried).
An alternative to consider if you want to play the media data stored in a file, is to store your media in a zip or rar since vlc has plugins to play media from directly inside such archives.
Just this, I'd like to stream uploaded filed directly from the net to the filesystem to avoid out of memory errors. Can I do it with CakeRequest::input()? Is there any other way?
Maybe it's best to read the API documentation CakeRequest::input() or the source;
http://api.cakephp.org/2.3/source-class-CakeRequest.html#876
According to the source, 'input()' reads directly from php://input via the _readInput() method:
However, if I read that part of the source code correctly, it will read the entire stream in memory before returning its content. So I don't think this will give you what you want.
There may be other solutions, maybe a plugin exists. However, you may write your own implementation, using the CakeRequest as an example?
You may also check the HttpSocket class
The bulk of the examples I can find for libxml2 are all about loading/parsing XML files. But I'm only interested in writing them; the code will never have to parse any files. There is an example using different writers, where it shows how to use the file, memory, DOM and tree models.
Looking through the code, I don't see any significant differences between them when it comes to writing. How does one decide which is better to use? (In other words, in what cases is one better than the others?)
The differences between the 4 functions you specify are minimal, it's all about where the contents go. As Alex mentioned, if memory is a concern, using xmlNewTextWriterFilename has the advantage of not needing to hold the result in memory.
The xmlWriter API, to which all the methods you mentioned belong, is one of the APIs offered. The other of note is the tree API. xmlWriter is more like calling write() to print to a file, and the tree is more like building nested structs in memory.
The tree-based versions can be good if your data is constructed in a non-linear fasion, going back and adding/changing things based on later information, etc. This would require some workarounds/caching with the streaming xmlWriter interface, as you can't change things once they've been output. The in-memory tree, however, can be fully tweaked until the instant it's serialized.
The tree API has the downside of the fact it has to keep the entire thing im memory; the rule of thumb is the memory requirements for a parsed tree is rougly 4x the size of serialized xml file.
My decision is usually dependent on whether I expect to create large documents. If not, I use the if the tree api, as the flexibility will be there if I want it. If I know efficiency will be a concern or I'll be working with large stuff, the streaming xmlWriter is the way to go.
tree API examples can be found here: http://xmlsoft.org/examples/index.html#Tree
If you're on a device with limited memory, you probably don't want to use DOM or memory-based approaches. In that case, you probably want to write out the file as you iterate through the data structure you want to write to XML.
First, a bit of context:
I'm trying to implement a URL shortening on my own server (in C, if that matters). The aim is to avoid long URLs while being able to restore a context from a shortened URL.
Currently I have a implementation that creates a session on the server, identified by a certain ID. This works, but consumes memory on the server (and is not desired since it's an embedded server with limited resources and the main purpose of the device isn't providing web pages but doing other cool stuff).
Another option would be to use cookies or HTML5 webstorage to store the session information in the client.
But what I'm searching for is the possibility to store the shortened URL parameters in one parameter that I attach to the URL and be able to re-construct the original parameters from that one.
First thought was to use a Base64-encoding to put all the parameters into one, but this produces an even larger URL.
Currently, I'm thinking of compressing the URL parameters (using some compression algorithm like zip, bz2, ...), do the Base64-encoding on that compressed binary blob and use that information as context. When I get the parameter, I could do a Base64-decoding, de-compress the result and have hands on the original URL.
The question is: is there any other possibility that I'm overlooking that I could use to lossless compress a large list of URL parameters into a single smaller one?
Update:
After the comments from home, I realized that I overlooked that compressing itself adds some overhead to the compressed data making the compressed data even larger than the original data because of the overhead that for example zipping adds to the content.
So (as home states in his comments), I'm starting to think that compressing the whole list of URL parameters is only really useful if the parameters are beyond a certain length because otherwise, I could end up having an even larger URL than before.
You can always roll your own compression. If you simply apply some huffman coding, the result will always be smaller (but then base64 encoding it, it'll grow a bit, so the net effect may perhaps not be optimal).
I'm using a custom compression strategy on an embedded project I work with where I first use a lzjb (a lempel ziv derivate, follow link for source code, really tight implementation (from open solaris)) followed by huffman coding the compressed result.
The lzjb algorithm doesn't perform too well on very short inputs, though (~16 bytes, in which case I leave it uncompressed).