how to read ZipInputStream into CharArrayReader - google-app-engine

Right now i am developing an application using Google App Engine (GAE). GAE doesnt allow me to create temp folder for me to store my zipfile and read from it. The only way is to read it from memory. The zipfile contains 6 CSV files which i need to read it into CSVReader.
//part of the code
MultipartFormDataRequest multiPartRequest = null;
Hashtable files = multiPartRequest.getFiles();
UploadFile userFile = (UploadFile)files.get("bootstrap_file");
InputStream input = userFile.getInpuStream();
ZipInputStream zin = new ZipInputStream(input);
How do i read ZipInputStream into char[] which is needed to create CharArrayReader for my CSVReader object.
CSVReader reader = new CSVReader(CharArrayRead(char[] buf));

Wrap the ZipInputStream with an InputStreamReader to convert from bytes to chars; then call inputStreamReader.read(char[] buf, int offset, int length) to fill your char[] buffer, like this:
//part of the code
MultipartFormDataRequest multiPartRequest = null;
Hashtable files = multiPartRequest.getFiles();
UploadFile userFile = (UploadFile)files.get("bootstrap_file");
InputStream input = userFile.getInpuStream();
ZipInputStream zin = new ZipInputStream(input);
// wrap the ZipInputStream with an InputStreamReader
InputStreamReader isr = new InputStreamReader(zin);
ZipEntry ze;
// ZipEntry ze gives you access to the filename etc of the entry in the zipfile you are currently handling
while ((ze = zin.getNextEntry()) != null) {
// create a buffer to hold the entire contents of this entry
char[] buf = new char[(int)ze.getSize()];
// read the contents into the buffer
isr.read(buf);
// feed the char[] to CSVReader
CSVReader reader = new CSVReader(CharArrayRead(buf));
}
if your CharArrayRead is actually a java.io.CharArrayReader, then there is no need to load it into a char[] and you'd be better off with code like this:
InputStreamReader isr = new InputStreamReader(zin);
BufferedReader br = new BufferedReader(isr);
ZipEntry ze;
while ((ze = zin.getNextEntry()) != null) {
CSVReader reader = new CSVReader(br);
}
If you just have a single zipped file (trying to get around the 1MB limitation) then this will work:
InputStreamReader isr = new InputStreamReader(zin);
zip.getNextEntry();
CSVReader reader = new CSVReader(isr, ...);

Related

Compress an existing string in DynamoDB table to a string : ZipException: Not in GZIP format

We want to compress a big string in our DynamoDB table, which is a JSON object.
I want to simply replace it with a compressed string. I looked into DynamoDB documentation, which uses ByteBuffer to be stored directly, as mentioned here.
But since I don't want to save ByteArray, and instead store a compressed string version of the original string, I have modified it.
Here is what I've done:
public class GZIPStringCompression {
public static String compress(String data) throws IOException {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(data.length());
GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream);
gzipOutputStream.write(data.getBytes());
gzipOutputStream.close();
return byteArrayOutputStream.toString();
}
public static String decompress(String compressed) throws IOException {
ByteArrayInputStream bis = new ByteArrayInputStream(compressed.getBytes());
GZIPInputStream gis = new GZIPInputStream(bis);
BufferedReader br = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
StringBuilder sb = new StringBuilder();
String line;
while((line = br.readLine()) != null) {
sb.append(line);
}
br.close();
gis.close();
bis.close();
return sb.toString();
}
}
This gives out the exception:
Exception in thread "main" java.util.zip.ZipException: Not in GZIP format
at java.base/java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)
at java.base/java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
at java.base/java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
at GZIPStringCompression.decompress(MyClass.java:41)
at MyClass.main(MyClass.java:16)
I am not sure what I want to is even possible, that's why, want to confirm that here.
Changed this to:
class GZIPStringCompression {
public static String compress(String data) throws IOException {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(data.length());
GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream);
gzipOutputStream.write(data.getBytes());
gzipOutputStream.close();
return Base64.getEncoder().encodeToString(byteArrayOutputStream.toByteArray());
}
public static String decompress(String compressed) throws IOException {
ByteArrayInputStream bis = new ByteArrayInputStream(Base64.getDecoder().decode(compressed));
GZIPInputStream gis = new GZIPInputStream(bis);
BufferedReader br = new BufferedReader(new InputStreamReader(gis, StandardCharsets.UTF_8));
StringBuilder sb = new StringBuilder();
String line;
while((line = br.readLine()) != null) {
sb.append(line);
}
br.close();
gis.close();
bis.close();
return sb.toString();
}
}
This somehow worked. Would this be a dependable solution?
‎Your first solution didn't work because you wanted to take a byte array (array of 8-bit bytes) and assign it to a String attribute (basically an array of unicode characters). That doesn't make sense can can result in all sorts of unwanted manipulation for your bytes that makes them unusable when you read them back.
Your approach of converting the byte array into base-64 encoding - basically a subset of ASCII - works, because ASCII characters can indeed be represented as in the String without any manipulation and can be read back just like they were written.
Since you mentioned this is for DynamoDB, I should add the DynamoDB does have the "binary" type in addition to the "string" type, and you could just use that. In Java, you can assign the byte array directly to an attribute with of this type - and don't need to try to "convert" it into a String.

Xamarin.android - Copy .jpg to shared folder

I'm using a Samba File Server for copy files from my Android device to a shared folder on Windows. With .txt files i haven't any problems, works fine, but I tried to copy a .jpg file into shared folder and it fails. I searched a lot of codes from internet but anyone solved my problem.
I managed to copy the image, but when I open it, is damaged.
Does anyone have any sample code?
My code is this:
Java.IO.File mfile = new Java.IO.File(item.FullName);
var mSharedFolderCalidad = new SmbFile(AppGlobalConfigSoftware.Instance.Parameters.PathToDownloadCalidad + item.Name);
//If exists don't create another time
if (!mSharedFolderCalidad.Exists())
mSharedFolderCalidad.CreateNewFile();
InputStream inFile = new FileInputStream(mfile);
SmbFileOutputStream sfos = new SmbFileOutputStream(mSharedFolderCalidad);
byte[] buf = new byte[1024];
int len;
while ((len = inFile.Read(buf)) > 0)
{
sfos.Write(buf, 0, len);
}
inFile.Close();
sfos.Close();
All help is appreciated.
Thank you.
You can use Media.Plugin from nuget to take photo firstly.
var file = await CrossMedia.Current.TakePhotoAsync(new StoreCameraMediaOptions
{
PhotoSize = PhotoSize.Medium,
});
public byte[] ReadFully(Stream input)
{
byte[] buffer = new byte[16*1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
MediaFile has GetStream().
You could use this to the Stream and then convert that to a byte[]. Here is one way to do that:
Define a stream
Stream imageStream;
And init it after you take the photo .
imageStream = file.GetStream();
var imageArr= ReadFully(imageStream );
And then write it to your folder .

Java - Load file as Template + append content in memory and manage byte[]

Im trying load a file in memory with a base information, append lines and include the result into a Zip file. In C# existes MemoryStream but, in java not.
the context of my application is load a stylesheet.css files with a pre-defined styles for add other styles that i get dinamically. Later i want add this content to a zip entry, and i need a byte[] that represent this content.
For the moment, i have the next lines, but i dont know as convert this to byte[]:
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
File file = new File(classLoader.getResource("style.css").getFile());
OutputStreamWriter osw = new OutputStreamWriter( new FileOutputStream( file ) );
BufferedWriter writer = new BufferedWriter(osw);
I tried, with ByteArrayOutputStream but i can't completed all my requirements.
Any idea? im opne to other ideas for get my goal. I looking for CSSParser too, but i didn't see as I can append content and get a File document (byte[]) for to add to my zip file.
Finnaly, i didn't find other solution for my problem that convert the InputStream to ByteArrayOutputStream byte to byte.
I created the following methods:
Load template file into Input Stream and convert.
private ByteArrayOutputStream getByteArrayOutputStreamFor(String templateName) {
try {
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
InputStream inStream = classLoader.getResourceAsStream( templateName ); //file into resources directory
ByteArrayOutputStream baos = Utils.toOutputStream(inStream);
return baos;
} catch (Exception e) {
String msg = String.format("erro to loaf filetemplate {%s}: %s", templateName, e.getMessage());
throw new RuntimeException( msg, e.getCause() );
}
}
Copy the inputStream into a ByteArrayOutputStream byte to byte
public static final ByteArrayOutputStream toOutputStream(InputStream inStream) {
try {
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
int byteReads;
while ((byteReads = inStream.read()) != -1) {
outStream.write(byteReads);
}
outStream.flush();
inStream.close();
return outStream;
} catch (Exception e) {
throw new RuntimeException("error message");
}
}
Finally, I append text to ByteArrayOutputStream
ByteArrayOutputStream baosCSS = getByteArrayOutputStreamFor( "templateName.css" );
BufferedWriter writer = new BufferedWriter( new OutputStreamWriter( baosCSS ) );
writer.append( "any text" );
writer.flush();
writer.close();
byte[] bytes = baosCSS.toByteArray()

System.IO.File.Copy Throws IOException

I am trying to copy some files:
private void DoCopy() {
string[] files = Directory.GetFiles(Application.StartupPath + "\\App_Data", "*.*", SearchOption.AllDirectories);
string sFtpToReadFileFrom = "ftp://<user>:<pass>#mysite.tk/updates/App_Data/";
string sPathToWriteFileTo = Application.StartupPath + "\\App_Data";
WebClient webClient = new WebClient();
webClient.Credentials = new NetworkCredential("user", "pass");
foreach (string s in files)
{
string fileName = Path.GetFileName(s);
string destFile = Path.Combine(sPathToWriteFileTo, fileName);
byte[] fileData = webClient.DownloadData(sFtpToReadFileFrom + fileName); //shows correct bytes
File.Copy(s, destFile, true);
}
}
The exact error is: The process cannot access the file 'C:\AppLauncher\AppLauncher\bin\Debug\App_Data\firstFile' because it is being used by another process.
I followed the 'MSDN How To' here: http://msdn.microsoft.com/en-us/library/cc148994.aspx
If anyone finds any immediate red flags please do let me know.
This is how I see it:
You want to download a file from an FTP server and write it to your local disk.
What you are doing is taking files in your source-directory as targets, which won't work at all. If the files already exist there, which they do if you can acquire their file names there. (hence the exception)
Here is what you must do
Connect to the FTP, acquire the files there (their bytes) and then create files on the disk.
private void DoCopy() {
//string[] files = Directory.GetFiles(Application.StartupPath + "\\App_Data", "*.*", SearchOption.AllDirectories);
//Acquire filenames from FTP-Server instead of local disk!
string sFtpToReadFileFrom = "ftp://<user>:<pass>#mysite.tk/updates/App_Data/";
string sPathToWriteFileTo = Application.StartupPath + "\\App_Data";
WebClient webClient = new WebClient();
webClient.Credentials = new NetworkCredential("user", "pass");
foreach (string s in files)
{
string fileName = Path.GetFileName(s); //create file names based on FTP-server
string destFile = Path.Combine(sPathToWriteFileTo, fileName);
byte[] fileData = webClient.DownloadData(sFtpToReadFileFrom + fileName); //shows correct bytes
//File.Copy(s, destFile, true); Rather use File.WriteAllBytes
File.WriteAllBytes(destFile, fileData);
}
}
See here for an example of File.WriteAllBytes.
The acquiring of the filenames from FTP is not that straightforward, though. There is the FtpWebRequest-class to support you.

save UDP socket data into file

I have a code to send and receive UDP socket
Send UDP code:
public static void main(String args[]) throws Exception
{
try
{
FileOutputStream fo = new FileOutputStream("OUTFILE.txt");
PrintStream ps = new PrintStream(fo);
DatagramSocket Socket = new DatagramSocket(4555);
byte[] receiveData = new byte[1000000];
DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);
while(true)
{
receivePacket.setLength(receiveData.length);
Socket.receive(receivePacket);
String sentence = new String( receivePacket.getData());
System.out.printf("RECEIVED: %s " , new String(receivePacket.getData()));
ps.println(sentence);
ps.println();
ps.close();
fo.close();
}
File file = new File("OUTFILE.txt");
FileInputStream fis = new FileInputStream(file);
byte[] fsize = new byte[(int) file.length()];
int size = fis.read(fsize);
System.out.println("Received Size = " + size);
}
catch (Exception e) {
System.err.println(e);
}
}
}
I want to write the value of each received data packet into file then get the size of the whole file.
In my code I just got the first received value written in the file.
Could you please tell me how can I write the whole received value in the file.
At the end of the first loop, you're closing the streams so additional lines can not be written. You need to move the ps.close(); and fs.close() calls to be outside of the loop. In addition, you have an indefinite loop which guarantees that the code reading from the file can not be called--you need to have some mechanism to determine when to stop looping.

Resources