I have a byte array how do I store it in Firestore?
preferable to store as a File in Firestore Storage but storing them as custom object is fine too in Cloud Firestore.
I tried to make a file and use the function write Bytes() and on retrieving use read bytes but that is not able to work proper and causing some disruption as my byte array is a encrypted form of image which I need to store in Firestore. And Retrieve later and decrypt it.
Before I was trying to convert Byte Array to string but many people told that is no good practice as it will cause loss of data.
Than when I researched a bit few people recommended to use Blob and store the byte array but unfortunately I was unable to get the sample code for it.
val z = cipher.doFinal(byteArray)
val file = File.createTempFile("$name", null)
file.writeBytes(z)
Log.d("first","quick")
val storageReference = FirebaseStorage.getInstance()
.getReference("/image/0XhL4jD4XCemk38rcRkIEjJMgjh2/Aadhar/")
storageReference.putFile(Uri.fromFile(file))
.addOnSuccessListener {
Toast.makeText(context , "Success", Toast.LENGTH_SHORT).show()
}
Basically Z is the encrypted byte array that I somehow need to push to Firestore. I tried to make file and push in storage. Its Successfully completed.
But when I retrieve it back Something like below:-
val storageRef =
FirebaseStorage.getInstance().reference?.child("/image/0XhL4jD4XCemk38rcRkIEjJMgjh2/Aadhar/")
val localfile = File.createTempFile("temp", ".txt")
storageRef.getFile(localfile)
.addOnSuccessListener {
val bytes = localfile.readBytes()
var decryptedText: ByteArray = cipher.doFinal(bytes)
val configBmp = Bitmap.Config.valueOf(bitmap.config.name)
val bitmap_tmp = Bitmap.createBitmap(width, height, configBmp)
val buffer = ByteBuffer.wrap(decryptedText)
bitmap_tmp.copyPixelsFromBuffer(buffer)
img.setImageBitmap(bitmap_tmp)
}
The problem when I read bytes I got no clue what but something is going wrong there. I can be of sure that encryption and decryption both are working fine as I tested it Locally without pushing the byte array to Firestore and Just decrypt and it works perfectly good.
But when for the same byte array I try to store in file and push and retrieve later it doesn't works. The error I get is Illegal block size.
Simplified Question:-
Assume I have a byte array 'X'
How can I store it in Firestore [Cloud Firestore / Storage in form of files[preferred] with 0 loss of data and retrieve it back.
Help would be highly appreciated guys with a snippet of code to understand,
Thanks in adv.
Related
I am testing out AES encryption on android and i need to convert the encrypted text to a string so as to store it in my firebase database. I managed to do so sucessfully. The problem comes when I try to retrieve the string from the database and convert it back to a byte array for decryption.
The retreived string is identical to the string sent to the database but when i convert it to a byte array and compare the two values, they are different as seen in the code sample below. I even made sure to use the same Charset but this did not fix the issue.
val encryptedPair: Pair<ByteArray,ByteArray> = encrypt("text")
val enc: MutableMap<String, Any> = HashMap()
enc["ciphertext"] = encryptedPair.second.toString(Charsets.UTF_8).trim()
fireStoreDatabase.collection("enc").document("test").set(enc)
.addOnSuccessListener {
fireStoreDatabase.collection("enc").document("test").get()
.addOnSuccessListener { doc->
//The below if condition prints out encrytion method: diff
if(encryptedPair.second.contentEquals(doc.getString("password")!!.toByteArray(Charsets.UTF_8)))
Log.e("encryption method: ","same")
else
Log.e("encryption method: ","diff")
}
}
This difference leads to the decryption process failing. Note that when i try to encrypt and decrypt with the values locally, it works as expected. So i know the problem is with the byte array i get from firebase.
Any suggestions would be appreciated.
As Jon Skeet has suggested, using
val enc: MutableMap<String, Any> = HashMap()
enc["ciphertext"] = Blob.fromBytes(encryptedPair.second)
fireStoreDatabase.collection("enc").document("test").set(enc)
to save the data to firebase and then retrieving it with:
doc.getBlob("ciphertext")!!.toBytes())
worked as expected.
I want to save small images in my room database and I have two issues:
How do I save an image in my database?
How do I save multiple images in my database?
I tried saving a bitmap in the way recommended by the developers page (https://developer.android.com/training/data-storage/room/defining-data)
#Parcelize
#Entity(tableName = "image_table")
data class ImgMod(
#PrimaryKey(autoGenerate = true)
var invoiceId: Long = 0L,
#ColumnInfo(name = "image")
var single_img: Bitmap?
): Parcelable
However, I receive the following error:
Cannot figure out how to save this field into database. You can consider adding a type converter for it.
Secondly, I would like to save multiple images in one database entry. But I receive the same error with the following snippets:
#ColumnInfo(name = "imageList")
var img_list: ArrayList<BitMap>
or decoded bitmap as a String
#ColumnInfo(name = "imageList")
var decoded_img_list: ArrayList<String>
I am sorry if this is a very basic question. But how do I have to configure the database/process the data to an image list?
Thank you in advance,
rot8
A very simple way to get images into the database (although I personally discourage it) would be to base-64 encode the bitmaps into a String and put it into a row of the database.
Take in account that bitmaps are very memory heavy; and base64 encoding something increases it's size some more so be careful when loading a bunch of images... I do also think Room and SQLite supports binary data as blobs, so you could just declare a column as ByteArray and it should just work.
What I've been doing in my projects is to write them into the internal or external storage of my app and then store the reference Uri as a String to later be able to retrieve the image from disk.
Something that I discourage even more is to stuff more than one value per row; having a list of stuff inside a coulmn in SQL is definetely not a pattern you should follow. Creating a "join" table should be simple enough or simply an extra column you could use to group them by should be easy enough, right?
I have a pyaudio stream running like so:
self.stream = self.microphone.open(format=pyaudio.paInt16, channels=1, rate=self.SAMPLING_RATE, input=True, output=True, frames_per_buffer=self.SAMPLES_PER_CHUNK)
and I am saving each chunk to an array after decoding it through numpy like so:
data = self.stream.read(self.SAMPLES_PER_CHUNK)
data = np.frombuffer(data, dtype='b')
recorded.append(list(data))
And I would later on like to be able to combine these chunks into a single array and save them to a wav file like this:
from scipy.io.wavfile import write
total = []
for i in recorded[start:stop]:
total += i # the issue is here
write('output2.wav', 48000, np.array(total).astype('int16'))
But clearly it is not as simple as merging the arrays as the output file is always just a snippet of static. Could someone tell me how I should be doing this?
I actually realized that it was a question of decoding the data which means that if you change this:
data = np.frombuffer(data, dtype='b')
To this:
data = np.frombuffer(data, dtype='int16')
The rest of the code works just fine
I'm experiencing a really weird situation when passing on a POJO java object within the payload of a Pull Queue task using Gson. Without changing the code or the POJO being set within the payload of a task, this will randomly succeed or fail.
This is the code I'm using:
PullQueueTaskPayLoad tqp = new PullQueueTaskPayLoad("id","name");
tqp.uploadURL = taskPayLoad.uploadURL;
tqp.urls = taskPayLoad.urls;
tqp.sliceQueryParameter = taskPayLoad.sliceQueryParameter;
TaskOptions task = TaskOptions.Builder.withMethod(TaskOptions.Method.PULL);
task.payload(new Gson().toJson(tqp));
q.add(task);
Using an external queue consumer I then retrieve the POJO as follows:
Type GSON_TYPE = new TypeToken<PullQueueTaskPayLoad>() {}.getType();
byte[] b = new Base64().decodeBase64(leasedTask.getPayloadBase64().getBytes());
String payload = new String(b);
logger.info("About to convert payload: "+payload);
PullQueueTaskPayLoad taskpayload = new Gson().fromJson(payload, GSON_TYPE);
So from the debugging I did, the problem seems to be happening when I'm decoding the payload bytes. While encoding the same POJO (with different Ids) I randomly get 2 different decoded payload Strings as follows:
Correct decoding:
{"id":"1786024566","sliceQueryParameter": {"queryId":786024566,"sliceStart":-1,"sliceNumber":1,"params":{"DefaultAnnotation":{"http://www.slicepedia.org/ontology#hasNumberOfBulletPoints_SIGN":["\u003d"],"http://www.slicepedia.org/ontology#hasNumberOfBulletPoints":["0"],"http://www.slicepedia.org/ontology#hasNumberOfTokens":["80"],"http://www.slicepedia.org/ontology#hasNumberOfTokens_SIGN":["\u003e"]},"VG":{"http://www.slicepedia.org/ontology#hastense":["?"],"http://www.slicepedia.org/ontology#hasroot":["?"]}}},"uploadURL":"http://3.linguabox0412.appspot.com/_ah/upload/AMmfu6YRjxX23Ks-yh-9AZs4-3I1p6hxrFd6d4ptxSQegUkQHN7y4hNZwX6u7PufIHJbwtsHLXFZJ5P-vs90mslZEOMw0T-amN2qhEOAj_6YdwuY50FXMi8/ALBNUaYAAAAAT7Towgs4M00M5RLI8xnEOMdIxouZzuGu/","status":"IN_PROGRESS","action":"SLICE_SEARCH_AND_CREATE"}
Incorrect decoding:
{"id":"1-1968382407","sliceQueryParameter":{"queryId":-1968382407,"sliceStart":-1,"sliceNumber":1,"params":{"DefaultAnnotation":{"http://www.slicepedia.org/ontology#hasNumberOfBulletPoints_SIGN":["\u003d"],"http://www.slicepedia.org/ontology#hasNumberOfBulletPoints":["0"],"http://www.slicepedia.org/ontology#hasNumberOfTokens":["80"],"http://www.slicepedia.org/ontology#hasNumberOfTokens_SIGN":["\u003e"]},"VG":{"http://www.slicepedia.org/ontology#hastense":["?K??????˜?X?\YXK?ܙ?????H?\ܛ????Ȃ%?????'W??EU$?#?&?GG???2?Ɩ?wV&??C"?7?B?6??????W??B???gSe????'u?U'd?D??6?S??4UV?D?e7?%U?&%F%f?D?$???$&vu6?fF$????EG?v??6?6դvt?D???G??&D?fdֵ6%?甦??GD????F???$?V?CuF?$?F?F֤֧f?D??u?wt?4?C$?W?"?'7FGW2#?$???$?u$U52"?&7F???#?%4Ĕ4U?4T$4???E?5$TDR'
So the second string obviously fails when using Gson to convert it back to a POJO. But I dont' understand why this happens in only some cases and not others. For what I've seen, it seems to always happen after a ["?"] character string. I tried replacing and ? with other strings but it didn't change anything.
I think what is happening here is that the payload is webSafe-base64 encoded. In practice, this means swapping + and / and = for - and _ and .. Most base64 libraries have native support for decoding websafe base 64 strings.
Probably you are meeting one of these chars at a certain point, and that kills the decoding.
Here is some info on WebSafeBase64
Word of warning, though: the taskqueue implementation is actually sending padding equals (=) that you will have to convert manually before parsing.
I want to save my Array's strucure and load it the next time I open my AIR application. Is there a way to store it to an EncryptedLocalStore item then get it later when I re-open the app?
EncryptedLocalStore.setItem() method takes a byte array when storing contents. To store an array, just use ByteArray.writeObject() method (as described in http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/flash/utils/ByteArray.html#writeObject()) to convert your Array to a ByteArray - and then persist the same to the ELS.
var array:Array = getArray();
var byteArray:ByteArray = new ByteArray();
byteArray.writeObject(array);
EncryptedLocalStore.setItem('somekey', byteArray);
Hope this helps.
Update: Added code to retrieve the array back.
var byteArray:ByteArray = EncryptedLocalStore.getItem('somekey');
var array:Array = byteArray.readObject() as Array;
Update: For custom classes.
In case you want to serialize your own custom classes to the ByteArray, you may have to call registerClassAlias() before writing the object to the ByteArray. For eg.
registerClassAlias("com.example.eg", ExampleClass);
I have found that it is easiest to to serialize the Array to a string and then store that string in the ELS. Then when you pull it out deserialize it back into an Array.