I'm using Kafka to send produce and consume messages.
Producing is fine, working with a <String, ByteArray> producer.
When consuming, I'm using the code below (taken from an example) but I'm getting each record as being just 8 bytes (sample output beneath code).
Is there a way a consumer can simply take a whole message as a byte array?
Code:
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.ByteArrayDeserializer");
KafkaConsumer<String, byte[]> consumer = new KafkaConsumer<String, byte[]>(props);
consumer.subscribe(Arrays.asList(topic));
int i = 0;
while (true) {
ConsumerRecords<String, byte[]> records = consumer.poll(100);
for (ConsumerRecord<String, byte[]> record : records)
System.out.printf("offset = %d, key = %s, value = %s\n", record.offset(), record.key(), record.value());
}
Output of System.out :
offset = 1773133, key = 105906453, value = [B#b8eff39
offset = 1773134, key = 105906453, value = [B#7bb1504
offset = 1773135, key = 105906453, value = [B#67b6c728
offset = 1773136, key = 105906453, value = [B#60b1f9c5
offset = 1773137, key = 105906177, value = [B#1cbab5dd
offset = 1773138, key = 105906177, value = [B#4376907b
offset = 1773139, key = 105906177, value = [B#122880ba
offset = 1773140, key = 105906177, value = [B#7db82ceb
offset = 1773141, key = 105906177, value = [B#34657adc
I'm not looking forward to having to assemble a load of these records to re-create a message as I believe I'm missing something and manual assembly may be error prone.
I think you should use
System.out.printf("offset = %d, key = %s, value = %s\n", record.offset(), record.key(), java.util.Arrays.toString(record.value()));
rather than depending on plain array.toString (which gives you garbage instead of actual content). Arrays are probably coming properly, you just debug it wrong way.
Related
I use AppDesigner inMATLAB to show photos with changed RGB. But there is the problem with character of the photo.
When I switch on my own fuction "changeRGB", finally "choosenImage" has 20bytes, class "char" and size(1x10). OK!
There is no problem in using the "function OpenButtonValueChanged". OK!
There is the problem with "function UploadButtonPushed". OK!
ABOUT THE PROBLEM:
When I click button which callback is "function UploadButtonPushed" I get the error:
"Error using imread>parse_inputs (line 502)
The file name or URL argument must be a character
vector or string scalar."
"Error in imread (line 342)
[source, fmt_s, extraArgs, was_cached_fmt_used] =
parse_inputs(cached_fmt, varargin{:});"
WHY?
Because in the "function UploadButtonPushed" my choosenImage has 1977624bytes, class "uint8" and size(681x968x3). So it's too bug for "imread".
WHAT I TRIED:
When in "function OpenButtonValueChanged" I convert a photo, adding "char": (myimage = char(app.clickedImage)); the class of the photo is changing from uint8 to char but the size.
When I use "num2cell", the claas of the photo is changing on "cell" but size and number of bytes are the same- so big. And I get the Error: "Error using imread>parse_inputs (line 502) The file name or URL argument must be a character vector or string scalar."
In my own function "changeRGB" I used "imread(image)" and here is the problem with the size of the photo. Do you know how to get the correct one?
%my own properties in AppDesigner- to use them in different functions
properties (Access = public)
clickedImage;
addR = 1;
addG = 1;
addB = 1;
end
%first function in AppDeesigner
function OpenButtonValueChanged(app, event)
value = app.OpenButton.Value;
[file, howManyFiles] = chooseImagesFromComputer; %myown function
%I load 3 images which are showed as miniatures
myFile1 = imread(file{1});
imshow(myFile1, 'Parent', app.UIAxes1_1);
myFile2 = imread(file{2});
imshow(myFile2, 'Parent', app.UIAxes1_2);
myFile3 = imread(file{3});
imshow(myFile3, 'Parent', app.UIAxes1_3);
%take values of changed RGB from the slider
app.addR = app.SliderR.Value
app.addG = app.SliderG.Value
app.addB = app.SliderB.Value
%work only on one image to change its colors. app.clickedImage, app.addR, app.addG, app.addB are properties at the beginning of the code
app.clickedImage = file{1};
app.clickedImage = changeRGB(app.clickedImage,app.addR,app.addG,app.addB); %changeRGB- my own function- here is the problem. I add it bottom
imshow(app.clickedImage,'Parent',app.UIAxesMain);
end
%second function in AppDesigner
%here is the button to upload color of the photo
function UploadButtonPushed(app, event)
myimage = app.clickedImage;
myimage = changeRGB(myimage,app.addR,app.addG,app.addB);
imshow(myimage);
end
%here is my own function in matlab, not in AppDesigner, which makes problem:
function [changedImage] = changeRGB(choosenImage, addR, addG, addB)
whos
loadedImage = imread(choosenImage);
R = loadedImage(:,:,1); %extract one of the color channels
G = loadedImage(:,:,2);
B = loadedImage(:,:,3);
RBG = cat(3,R,G,B);
R_adj2 = R + addR;
G_adj2 = G + addG;
B_adj2 = B + addB;
changedImage = cat(3,R_adj2,G_adj2,B_adj2);
end
First, you make unnecessary operations in changeRGB
function [changedImage] = changeRGB(choosenImage, addR, addG, addB)
loadedImage = imread(choosenImage);
loadedImage = bsxfun(#sum, loadedImage, reshape([addR, addG, addB], [1 1 3]);
end
Then this function return an array (the modified image) so in UploadButtonPushed(app, event) when you run myimage = app.clickedImage;, you are passing the modified array instead of the image path, that you set here app.clickedImage = changeRGB(app.clickedImage,app.addR,app.addG,app.addB);
So you have to change the design of your variables, because app.clickedImage is saving either the image path, or the image itself. Consider having 2 different variables.
A good advice also is to use matlab debugger which is really good help to find the source of this kind of problems.
I want to insert the below default values when i am running the service i got this below error any one please tell me how to resolve.
Runtime error in script ("Process: 'CustomPersonalGS Practice' ProcessItem: 'Initialize' Type: 'ITEM'" -1:-1).TypeError: Cannot read property "parameters" from null
//Initialise SQL Query List
tw.local.sqlQueries = new tw.object.listOf.SQLStatement();
tw.local.sql = "";
tw.local.customerPD = new tw.object.customerPD1BO();
tw.local.customerPD.customerPersonalDetailsList = new tw.object.listOf.customerSpecificPersonalDetailsListBO();
var custPersonalDetails = new tw.object.customerSpecificPersonalDetailsListBO();
custPersonalDetails.customerId = "8467";
custPersonalDetails.custPersonalDetailsId = "8";
custPersonalDetails.isBPMEnabled = true;
custPersonalDetails.isCCPEnabled = true;
custPersonalDetails.isCCPMandatory = true;
custPersonalDetails.isLatestVersion = true
tw.local.customerPD.customerPersonalDetailsList.insertIntoList(tw.local.customerPD.customerPersonalDetailsList.listLength, custPersonalDetails);
tw.local.sql = "INSERT INTO CUSTOMPERSONALDETAILSQUESTION(CUSTOMERID,CUSTPERSONLADETAILSID,ISBPMENABLED,ISCCPENABLED,ISCCPMANDATORY,ISLATESTVERSION) VALUES (?,?,?,?,?,?) ";
function addSQLStatement() {
tw.local.sqlQueries[tw.local.sqlQueries.listLength] = new tw.object.SQLStatement();
}
function addParam(value,type,mode) {
log.info("VALUE :" + value);
var newParam = new tw.object.SQLParameter();
newParam.value = value;
newParam.type = type;
newParam.mode = mode;
if( tw.local.sqlQueries == null){
tw.local.sqlQueries = new tw.object.listOf.SQLStatement();
}
if( tw.local.sqlQueries[tw.local.sqlQueries.listLength] == null ){
tw.local.sqlQueries.insertIntoList(tw.local.sqlQueries.listLength, new tw.object.SQLStatement());
}
if(tw.local.sqlQueries[tw.local.sqlQueries.listLength].parameters == null ){
tw.local.sqlQueries[tw.local.sqlQueries.listLength].parameters = new tw.object.listOf.SQLParameter();
}
var paramsLength = tw.local.sqlQueries[tw.local.sqlQueries.listLength].parameters.listLength;
tw.local.sqlQueries[tw.local.sqlQueries.listLength].parameters[paramsLength] = newParam;
}
for(var i=0;i<tw.local.customerPD.customerPersonalDetailsList.listLength;i++){
addSQLStatement(tw.local.sql);
addParam(tw.local.customerPD.customerPersonalDetailsList[i].customerId,"VARCHAR","IN");
addParam(tw.local.customerPD.customerPersonalDetailsList[i].custPersonalDetailsId,"VARCHAR","IN");
var yesNoFlag = "N";
if(tw.local.customerPD.customerPersonalDetailsList[i].isBPMEnabled){
yesNoFlag="Y";
addParam(yesNoFlag,"CHAR","IN");
}
yesNoFlag = "N";
if(tw.local.customerPD.customerPersonalDetailsList[i].isCCPEnabled){
yesNoFlag="Y";
addParam(yesNoFlag,"CHAR","IN");
}
yesNoFlag = "N";
if(tw.local.customerPD.customerPersonalDetailsList[i].isCCPMandatory){
yesNoFlag="Y";
addParam(yesNoFlag,"CHAR","IN");
}
yesNoFlag = "N";
if(tw.local.customerPD.customerPersonalDetailsList[i].isLatestVersion){
yesNoFlag="Y";
addParam(yesNoFlag,"CHAR","IN");
}
}
You didn't initialize the parameter list in your SQL as far as I can tell. That is on line 38 you call -
var paramsLength = tw.local.sqlQueries[tw.local.sqlQueries.listLength].parameters.listLength;
However when you create the entry in the tw.local.sqlQueries, you didn't initialize the parameter array. I'll also note that your addSQLStatement() function ignore the sql input (and that value is hard coded, so really you don't need to pass it in). I think if you change addSQLStatement to be something like -
function addSQLStatement(query) {
var targetQuery = new tw.object.SQLStatement();
targetQuery.sql = query;
tagetQuery.params = new tw.object.listOf.SQLParameter();
tw.local.sqlQueries[tw.local.sqlQueries.listLength] = targetQuery;
}
then your code will work. Additionally you could actually return targetQuery from this function and then pass it to the "addParams" method eliminating the need to find the last one in the array. Alternatively insert it in the beginning of the array instead and just update the 0th item instead of the last.
-AP
This comparison will never work properly. array[array.length] will always be null (line 35).
if (tw.local.sqlQueries[tw.local.sqlQueries.listLength].parameters == null ){
In addition, in the next lines, if you want to work with the last element of the list, you might want to use something like array[array.length - 1]. Personally, I'd use some temporary variable, do some stuff with it and insert it into the list at the end (similar to #Drux's answer).
using the randdusing bluetoothle plugin for ionic app, need to read the advertisement.
The ble scan returns with Start Scan Success :
{"address":"14::30:c6:60:e8;9f","name":null,"rssi":-50,"advertisement":"AgEGG/9SVgIADSw5YTNlMTQAAAJlAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA;=","status":"scanResult"}
query: need to decipher this json data and convert this advertisement data into array containing hex values of advertisement data? The advertisement data seems to be base64 encoded. Please advice.
I made for this purpose a little helper function as shown below. The key is the $cordovaBluetoothLE.encodedStringToBytes as you can see in docs https://github.com/randdusing/ng-cordova-bluetoothle.
var encodedToByteString = function encodedToByteString(input) {
var val = $cordovaBluetoothLE.encodedStringToBytes(input);
var byteStr = "";
for (var i = 0; i < val.length; i++) {
var byte = val[i].toString(16);
if (byte.length == 1) byte = "0" + byte;
byteStr += byte;
}
return byteStr;
};
The same goes for the opposite operation - that is sending data. You first need to get your hex-string into an array of bytes and then encode it via $cordovaBluetoothLE.bytesToEncodedString(value).
I'm stuck with the BIP70 documentation. I can't figure out how to translate the repeated fields on that object.
message PaymentDetails {
optional string network = 1 [default = "main"];
repeated Output outputs = 2;
required uint64 time = 3;
optional uint64 expires = 4;
optional string memo = 5;
optional string payment_url = 6;
optional bytes merchant_data = 7;
}
With the bitcore-payment-protocol npm package, here is how they fill the outputs filled in the docs.
var details = new PaymentProtocol().makePaymentDetails();
details.set('network', 'test');
details.set('outputs', outputs);<=== shall ouputs be an array?
details.set('time', now);
details.set('expires', now + 60 * 60 * 24);
details.set('memo', 'A payment request from Ambrosia.');
details.set('payment_url', 'https://localhost/-/pay');
details.set('merchant_data', new Buffer({size: 7}));
Here is my question, I want more than one output, should I fill it with an array of output object?
I'm very new to DSP. And have to solve the following problem: applying the low shelving filter for an array of data. The original data is displayed in fract16 (VisualDSP++).
I'm writing something as below but not sure it's correct or not.
Does the following code have any problem with overflow?
If 1 is true, how should I do to prevent it?
Any advice on this problem?
fract16 org_data[256]; //original data
float16 ArrayA[],ArrayB[];
long tmp_A0, tmp_A1, tmp_A2, tmp_B1, tmp_B2;
float filter_paraA[3], filter_paraB[3]; // correctness: 0.xxxxx
// For equalizing
// Low-Shelving filter
for ( i=0; i<2; i++)
{
tmp_A1 = ArrayA[i*2];
tmp_A2 = ArrayA[i*2+1];
tmp_B1 = ArrayB[i*2];
tmp_B2 = ArrayB[i*2+1];
for(j=0;j<256;j++){
tmp_A0 = org_data[j];
org_data[j] = filter_paraA[0] * tmp_A0
+ filter_paraA[1] * tmp_A1
+ filter_paraA[2] * tmp_A2
- filter_paraB[1] * tmp_B1
- filter_paraB[2] * tmp_B2;
tmp_A2 = tmp_A1;
tmp_B2 = tmp_B1;
tmp_A1 = tmp_A0;
tmp_B1 = org_data[j];
}
ArrayA[i*2] = tmp_A1;
ArrayA[i*2+1] = tmp_A2;
ArrayB[i*2] = tmp_B1;
ArrayB[i*2+1] = tmp_B2;
}
I don't know what the range is for fract16, just -1 to +1 approx?
The section that stands out to me as possibly generating an overflow is assigning org_data[j] but will be dependent on what you know about your input signal and your filter coefficients. If you can ensure that multiplying filter_paraA[2:0] to signal with values tmp_A2..1 = [1,1,1] is < max(fract16) you should be fine regardless of the 'B' side.
I would recommend adding some checks for overflow in your code. It doesn't necessarily have to fix it, but you would be able to identify an otherwise very tricky bug. Unless you need absolute max performance I would even leave the check code in place but with less output or setting a flag that gets checked.
macA = filter_paraA[0] * tmp_A0 + filter_paraA[1] * tmp_A1 \
+ filter_paraA[2] * tmp_A2;
macB = filter_paraB[1] * tmp_B1 - filter_paraB[2] * tmp_B2;
if((macA-macB)>1){
printf("ERROR! Overflow detected!\n");
printf("tmp_A[] = [%f, %f, %f]\n",tmp_A2,tmp_A1,tmp_A0);
printf("tmp_B[] = [%f, %f]\n",tmp_B1,tmp_B0);
printf(" i = %i, j = %i\n",i,j);
}