I have a java serializable object and I want to write/read that object to/from a file.
Yet Google app engine does not support fileoutputstream and objectoutputstream so I convert the serializable object to a byte array and write it to file
$FileWriteChannel writeChannel = fileService.openWriteChannel(
writableFile, lockForWrite);
$writeChannel.write(ByteBuffer.wrap(bytearray));
$writeChannel.closeFinally();
Then I read the file:
AppEngineFile readableFile = new AppEngineFile(filename);
FileReadChannel readChannel = fileService.openReadChannel(readableFile, lockForRead);
BufferedReader reader = new BufferedReader(Channels.newReader(readChannel, "UTF-8"));
String line = reader.toString();
I see the log and I got error with UTF-8
Can anybody show me what I did incorrectly ?
Error log:
cloud.spam.filter.server.SerializationObject file_io: An error message: invalid stream header: 5B4240312012/07/06 08:00:10
Thank you
Consider using protobuf for this purpose. Beside the fact that it's more compact format than standard java serialization, it will also give you ability to change (by adding new fields) class to be written into file.
Related
Help me please, I can't get access to file which I choose by FileOpenPicker.
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.ViewMode = PickerViewMode.Thumbnail;
openPicker.SuggestedStartLocation = PickerLocationId.Desktop;
openPicker.CommitButtonText = "Открыть";
openPicker.FileTypeFilter.Add(".xlsx");
var file = await openPicker.PickSingleFileAsync();
using (FileStream fs = new FileStream(file.Path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
}
What is wrong?
Because of how UWP sandboxes access to the filesystem, you can't construct a FileStream directly from a StorageFile's path. Instead, you have a few options, in order from simplest to most complex:
1) If your file is small enough, you can just use the helpers in the FileIO static class to read it all at once:
string text = await FileIO.ReadTextAsync(file); // or ReadLinesAsync or ReadBufferAsync, depending on what you need
2) Use the OpenAsync() method on StorageFile:
using (IRandomAccessStream stream = await file.OpenAsync(FileAccessMode.Read, StorageOpenOptions.AllowReadersAndWriters))
{
// your reading code here
}
If you need to, you can convert between IRandomAccessStream and .NET Streams with the AsStream(), AsStreamForRead() and AsStreamForWrite() extension methods on IRandomAccessStream, the docs for which are here.
3) If you want complete control, you can get a SafeFileHandle to the underlying file using CreateSafeFileHandle(), like so:
SafeFileHandle fileHandle = file.CreateSafeFileHandle(FileAccess.Read, FileShare.ReadWrite);
You can then use this file handle to create a standard FileStream:
using (FileStream fs = new FileStream(fileHandle, FileAccess.Read))
{
// Read stuff here
}
This is the only way to reliably use a FileStream on a UWP StorageFile, and should be used with some caution. The official docs have more details on the implications of doing this.
FileOpenPicker gives you a StorageFile which wraps the opened files and gives you permission to it. This doesn't give you access to the file in general - e.g. you cannot use its Path only to open it with a FileStream. Instead, you need to use the respective Windows.Storage APIs to do this. I usually use the OpenStreamForReadAsync extension method. Add using System.IO to the file header and then:
var stream = await file.OpenStreamForReadAsync();
This method returns a System.IO.Stream which you can use with classic System.IO-enabled APIs.
Please avoid use file Path to access file stream in UWP platform, if you have get the file with FileOpenPicker. You could get the file stream with following.
FileOpenPicker openPicker = new FileOpenPicker();
openPicker.ViewMode = PickerViewMode.Thumbnail;
openPicker.SuggestedStartLocation = PickerLocationId.Desktop;
openPicker.CommitButtonText = "Открыть";
openPicker.FileTypeFilter.Add(".xlsx");
var file = await openPicker.PickSingleFileAsync();
if (file != null)
{
var stream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read);
}
And if you need use stream in System.IO namespace. please call AsStream method for IRandomAccessStream object.
var stream = await file.OpenAsync(Windows.Storage.FileAccessMode.Read);
System.IO.Stream iostream = stream.AsStream();
For more detail please refer this official document.
I'm a little confused around the issue of returning a byte array vs a stream in an HTTP Response using .net Web API.
I came across the following code:
SqlConnection conn = new SqlConnection();
SqlCommand cmd = conn.CreateCommand();
cmd.CommandText = "Select FileData.PathName() As FilePath, GET_FILESTREAM_TRANSACTION_CONTEXT() AS Context From FileStorage";
conn.Open();
SqlDataReader reader = cmd.ExecuteReader();
reader.Read();
string filePath = (string)reader["FilePath"];
byte[] fileBytes = (byte[])reader["Context"];
SqlFileStream stream = new SqlFileStream(filePath, fileBytes, FileAccess.Read);
result.Content = new StreamContent(stream);
result.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
Question 1:
Why would they return a Stream instead of a byte array in the HTTP Response?
Question 2:
Why create a SqlFileStream to read the data if the byte array is already available by calling (byte[])reader["Context"]? Wouldn't this mean that the entire file contents are read into memory? So why the need for a Stream?
Question 1: Why would they return a Stream instead of a byte array in the HTTP Response?
Because the byte array may be huge, so if you read the entire array into the memory of the server and keep it in memory until it has all been transmitted to the client you are imposing a huge memory burden on the server. That's the stuff Denial-Of-Service attacks are made of. By using a stream you allow the server to load the data in small chunks, on an as-needed basis, and to keep only a small chunk in memory at any given time, while waiting for it to be transmitted.
Question 2: Why create a SqlFileStream to read the data if the byte array is already available by calling (byte[])reader["Context"]? Wouldn't this mean that the entire file contents are read into memory? So why the need for a Stream?
The byte array that you see there is not the actual file contents. If you look at the documentation of the constructor of SqlFileStream, and also at the documentation of the SqlFileStream class, this byte array is some "transaction context" which is (a terrible hack) necessary for the database server to read the actual the data from storage. The actual data is potentially huge, so the code that you posted does all this in order to avoid loading it all into memory.
Buffering is the main reason for returning StreamContent. In ASP.NET Web API every time you return StreamContent, your response is not buffered however byte array response is already buffered and available to serve. In the case of byte[] the content of HttpResponseMessage could be set directly from your byte[] and you do not need to convert it to Stream type.
In addition consider using PushStreamContent in scenarios in which you want to stream binary contents to the client continuously so client can consume your api progressively as the data arrives similar to following code snipet:
var httpResponseMessage = new HttpResponseMessage
{
Content = new PushStreamContent(async (respStream, content, context) =>
{
using (var writer = new StreamWriter(respStream))
{
await writer.WriteLineAsync();
await writer.FlushAsync();
}
}, "text/plain")
};
I'm using the Gmail API in browser and want to allow the user to download email attachments. I see https://developers.google.com/gmail/api/v1/reference/users/messages/attachments/get but it returns JSON and base64 data. I don't think I can get that data in memory then trigger a "download" to save the file locally. Even if I could I don't think it would be efficient - it would probably download the file in memory vs. streaming it to a file. I think I need a direct link to a file that returns the correct file name and raw binary data (not base64). Is there a way to do this? Right now the only way I see is to proxy requests.
You can get the data from the base64 and save it to file locally.
If you are getting the attachment in Java, you can use the FileOutputStream class (or f.write() in Python) to write the bytes to file and save it locally with a path.
You can try with the following sample code from Google Developer page:
public static void getAttachments(Gmail service, String userId, String messageId)
throws IOException {
Message message = service.users().messages().get(userId, messageId).execute();
List<MessagePart> parts = message.getPayload().getParts();
for (MessagePart part : parts) {
if (part.getFilename() != null && part.getFilename().length() > 0) {
String filename = part.getFilename();
String attId = part.getBody().getAttachmentId();
MessagePartBody attachPart = service.users().messages().attachment().
get(userId, messageId, attId).execute();
byte[] fileByteArray = Base64.decodeBase64(attachPart.getData());
FileOutputStream fileOutFile =
new FileOutputStream("directory_to_store_attachments" + filename);
fileOutFile.write(fileByteArray);
fileOutFile.close();
}
}
}
I did a lot of research and came up with the code below. It successfully translates speech to text using the microphone.
I have a file on my webserver that streams audio via mp3. It is just a link to an mp3 file. I need to have that translated to text.
I am trying to figure out the best way to do this. So, can you select the audio input as the computers audio (ie play the audio in the web browser)? Or can you stream the audio directly to the translator? I think I need to use SetInputToWaveStream method, but do not understand how to use it.
Private Sub InitializeRecognizerSynthesizer()
Dim selectedRecognizer = ( _
Where e.Culture.Equals(Thread.CurrentThread.CurrentCulture)).FirstOrDefault()
recognizer = New SpeechRecognitionEngine(selectedRecognizer)
recognizer.AudioStateChanged += New EventHandler(Of AudioStateChangedEventArgs)(recognizer_AudioStateChanged)
recognizer.SpeechHypothesized += New EventHandler(Of SpeechHypothesizedEventArgs)(recognizer_SpeechHypothesized)
recognizer.SpeechRecognized += New EventHandler(Of SpeechRecognizedEventArgs)(recognizer_SpeechRecognized)
synthesizer = New SpeechSynthesizer()
End Sub
Private Function SelectInputDevice() As Boolean
Dim proceedLoading As Boolean = True
If IsOscompatible() Then
Try
recognizer.SetInputToDefaultAudioDevice()
Catch
'no audio input device
proceedLoading = False
End Try
Else
ThreadPool.QueueUserWorkItem(InitSpeechRecogniser)
End If
Return proceedLoading
End Function
recognizer.SetInputToWaveFile(file) - will read the audio input from a file in the file system.
recognizer.SetInputToAudioStream - will read the audio input from a stream. A short example:
FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
SpeechAudioFormatInfo format = new SpeechAudioFormatInfo(8000, AudioBitsPerSample.Sixteen, AudioChannel.Mono);
recognizer.SetInputToAudioStream(fs, format);
When reading from a stream or a file you must use care to make sure that the audio data is in a supported format. For example, one format I know works on my machine is:
8 bits per sample
single channel mono
22,050 samples per second
PCM encoding
See Help with SAPI v5.1 SpeechRecognitionEngine always gives same wrong result with C# for more info about audio formats.
If your question is how to fetch a resource from a web server and handle it as a stream, see HttpWebResponse.GetResponseStream - http://msdn.microsoft.com/en-us/library/system.net.httpwebresponse.getresponsestream(v=vs.100).aspx
I am using Restlet framework. I create one web service that returns a file to the client.
On the server side, I first create a FileRepresentation object, instantiate it correctly, and return it to the client as Representation.
On the client side, I want to extract the content of the Representation, how can I cast the Representation object to FileRepresentation?
Thanks in advance!!
In fact, the FileRepresentation class is provided in order to fill request / response from a file but can't be used to extract content of a response.
To have access to your response content on the client side, it depends on the file type. If you receive an ascii content, you can do something like that:
Representation representation = resource.get();
String fileContent = representation.getText();
If it's a binary file, you need to work with a stream, as described below:
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
resource.get().write(outputStream);
byte[] fileContent = outputStream.toByteArray();
Hope it helps you,
Thierry