Deserialize multiple ArrayLists in a class using gson - arrays

I have a class with multiple Lists. Is the class design correct for gson to work?
My class:
Class Data
{
List<String> actor;
List<Integer> text_relevance;
List<String> title;
}
The json string is as follows ...
Is the class design for this json correct?
{
"data":
{
"actor":["Abercrombie, Ian","Baker, Dee Bradley","Burton, Corey",
"Eckstein, Ashley","Futterman, Nika","Kane, Tom",
"Lanter, Matt","Taber, Catherine","Taylor, James Arnold",
"Wood, Matthew"],
"text_relevance":["308"],
"title":["Star Wars: The Clone Wars"]
}
}

Yes, your code is correct. I deserialized it with this code:
Gson gson = new Gson();
Type type = new TypeToken<Map<String, Data>>() {}.getType();
Map<String, Data> map = gson.fromJson(json, type);
Data data = map.get("data");
Note that Gson will convert strings like "308" to the corresponding integer. This is generally used to permit very large values from losing precision. But it works out just fine for your case as well.

Related

is JSONDeserializationSchema() deprecated in Flink?

I am new to Flink and doing something very similar to the below link.
Cannot see message while sinking kafka stream and cannot see print message in flink 1.2
I am also trying to add JSONDeserializationSchema() as a deserializer for my Kafka input JSON message which is without a key.
But I found JSONDeserializationSchema() is not present.
Please let me know if I am doing anything wrong.
JSONDeserializationSchema was removed in Flink 1.8, after having been deprecated earlier.
The recommended approach is to write a deserializer that implements DeserializationSchema<T>. Here's an example, which I've copied from the Flink Operations Playground:
import org.apache.flink.api.common.serialization.DeserializationSchema;
import org.apache.flink.api.common.typeinfo.TypeInformation;
import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper;
import java.io.IOException;
/**
* A Kafka {#link DeserializationSchema} to deserialize {#link ClickEvent}s from JSON.
*
*/
public class ClickEventDeserializationSchema implements DeserializationSchema<ClickEvent> {
private static final long serialVersionUID = 1L;
private static final ObjectMapper objectMapper = new ObjectMapper();
#Override
public ClickEvent deserialize(byte[] message) throws IOException {
return objectMapper.readValue(message, ClickEvent.class);
}
#Override
public boolean isEndOfStream(ClickEvent nextElement) {
return false;
}
#Override
public TypeInformation<ClickEvent> getProducedType() {
return TypeInformation.of(ClickEvent.class);
}
}
For a Kafka producer you'll want to implement KafkaSerializationSchema<T>, and you'll find examples of that in that same project.
To solve the problem of reading non-key JSON messages from Kafka I used case class and JSON parser.
The following code makes a case class and parses the JSON field using play API.
import play.api.libs.json.JsValue
object CustomerModel {
def readElement(jsonElement: JsValue): Customer = {
val id = (jsonElement \ "id").get.toString().toInt
val name = (jsonElement \ "name").get.toString()
Customer(id,name)
}
case class Customer(id: Int, name: String)
}
def main(args: Array[String]): Unit = {
val env = StreamExecutionEnvironment.getExecutionEnvironment
val properties = new Properties()
properties.setProperty("bootstrap.servers", "xxx.xxx.0.114:9092")
properties.setProperty("group.id", "test-grp")
val consumer = new FlinkKafkaConsumer[String]("customer", new SimpleStringSchema(), properties)
val stream1 = env.addSource(consumer).rebalance
val stream2:DataStream[Customer]= stream1.map( str =>{Try(CustomerModel.readElement(Json.parse(str))).getOrElse(Customer(0,Try(CustomerModel.readElement(Json.parse(str))).toString))
})
stream2.print("stream2")
env.execute("This is Kafka+Flink")
}
The Try method lets you overcome the exception thrown while parsing the data
and returns the exception in one of the fields (if we want) or else it can just return the case class object with any given or default fields.
The sample output of the Code is:
stream2:1> Customer(1,"Thanh")
stream2:1> Customer(5,"Huy")
stream2:3> Customer(0,Failure(com.fasterxml.jackson.databind.JsonMappingException: No content to map due to end-of-input
at [Source: ; line: 1, column: 0]))
I am not sure if it is the best approach but it is working for me as of now.

Map<String ,String[]> Converts to Map<String, Object[])

I have a map
Map<String, String[]> newMetaData = new LinkedHashMap();
which I populate with data like this:
newMetaData.put(
((String) elm.get("companyName")).trim(),
new String[]{
this.storeFile( "logo", (String) elm.get("logoLink") ),
this.storeFile( "profile", (String) elm.get("companyProfile") ),
this.storeFile( "action", (String) elm.get("drash") ),
(String) elm.get("fwtografies")
}
);
StoreFile is a function that returns string. I save this map to the storage as
Storage.getInstance().writeObject("MetaData", newMetaData);
Later on the code I retrive the above map as:
Map<String, String[]> metaData = (Map)Storage.getInstance().readObject("MetaData");
But instead of getting a Map of <String, String[] & gt; I get Map of <String, Object[] >
any help is appreciated
Generics in Java are syntax sugar. They are erased by the compiler and ignored by Codename One for the most part. This:
Map<String, String[]> newMetaData = new LinkedHashMap<>();
Is equivalent to this as far as the generated code is concerned:
Map newMetaData = new LinkedHashMap();
There are some caveats because the first version will also add casts when you use methods like set/put which could trigger a class cast exception in some cases. The thing is that this works in reverse as well so this should work just fine:
Map<String, String[]> metaData = (Map<String, String[]>)Storage.getInstance().readObject("MetaData");

Why Spring is turning my object into an array of attributes? [duplicate]

I'm developing a Spring Boot application with Spring Data JPA. I'm using a custom JPQL query to group by some field and get the count. Following is my repository method.
#Query(value = "select count(v) as cnt, v.answer from Survey v group by v.answer")
public List<?> findSurveyCount();
It's working and result is obtained as follows:
[
[1, "a1"],
[2, "a2"]
]
I would like to get something like this:
[
{ "cnt":1, "answer":"a1" },
{ "cnt":2, "answer":"a2" }
]
How can I achieve this?
Solution for JPQL queries
This is supported for JPQL queries within the JPA specification.
Step 1: Declare a simple bean class
package com.path.to;
public class SurveyAnswerStatistics {
private String answer;
private Long cnt;
public SurveyAnswerStatistics(String answer, Long cnt) {
this.answer = answer;
this.count = cnt;
}
}
Step 2: Return bean instances from the repository method
public interface SurveyRepository extends CrudRepository<Survey, Long> {
#Query("SELECT " +
" new com.path.to.SurveyAnswerStatistics(v.answer, COUNT(v)) " +
"FROM " +
" Survey v " +
"GROUP BY " +
" v.answer")
List<SurveyAnswerStatistics> findSurveyCount();
}
Important notes
Make sure to provide the fully-qualified path to the bean class, including the package name. For example, if the bean class is called MyBean and it is in package com.path.to, the fully-qualified path to the bean will be com.path.to.MyBean. Simply providing MyBean will not work (unless the bean class is in the default package).
Make sure to call the bean class constructor using the new keyword. SELECT new com.path.to.MyBean(...) will work, whereas SELECT com.path.to.MyBean(...) will not.
Make sure to pass attributes in exactly the same order as that expected in the bean constructor. Attempting to pass attributes in a different order will lead to an exception.
Make sure the query is a valid JPA query, that is, it is not a native query. #Query("SELECT ..."), or #Query(value = "SELECT ..."), or #Query(value = "SELECT ...", nativeQuery = false) will work, whereas #Query(value = "SELECT ...", nativeQuery = true) will not work. This is because native queries are passed without modifications to the JPA provider, and are executed against the underlying RDBMS as such. Since new and com.path.to.MyBean are not valid SQL keywords, the RDBMS then throws an exception.
Solution for native queries
As noted above, the new ... syntax is a JPA-supported mechanism and works with all JPA providers. However, if the query itself is not a JPA query, that is, it is a native query, the new ... syntax will not work as the query is passed on directly to the underlying RDBMS, which does not understand the new keyword since it is not part of the SQL standard.
In situations like these, bean classes need to be replaced with Spring Data Projection interfaces.
Step 1: Declare a projection interface
package com.path.to;
public interface SurveyAnswerStatistics {
String getAnswer();
int getCnt();
}
Step 2: Return projected properties from the query
public interface SurveyRepository extends CrudRepository<Survey, Long> {
#Query(nativeQuery = true, value =
"SELECT " +
" v.answer AS answer, COUNT(v) AS cnt " +
"FROM " +
" Survey v " +
"GROUP BY " +
" v.answer")
List<SurveyAnswerStatistics> findSurveyCount();
}
Use the SQL AS keyword to map result fields to projection properties for unambiguous mapping.
This SQL query return List< Object[] > would.
You can do it this way:
#RestController
#RequestMapping("/survey")
public class SurveyController {
#Autowired
private SurveyRepository surveyRepository;
#RequestMapping(value = "/find", method = RequestMethod.GET)
public Map<Long,String> findSurvey(){
List<Object[]> result = surveyRepository.findSurveyCount();
Map<Long,String> map = null;
if(result != null && !result.isEmpty()){
map = new HashMap<Long,String>();
for (Object[] object : result) {
map.put(((Long)object[0]),object[1]);
}
}
return map;
}
}
I know this is an old question and it has already been answered, but here's another approach:
#Query("select new map(count(v) as cnt, v.answer) from Survey v group by v.answer")
public List<?> findSurveyCount();
define a custom pojo class say sureveyQueryAnalytics and store the query returned value in your custom pojo class
#Query(value = "select new com.xxx.xxx.class.SureveyQueryAnalytics(s.answer, count(sv)) from Survey s group by s.answer")
List<SureveyQueryAnalytics> calculateSurveyCount();
I do not like java type names in query strings and handle it with a specific constructor.
Spring JPA implicitly calls constructor with query result in HashMap parameter:
#Getter
public class SurveyAnswerStatistics {
public static final String PROP_ANSWER = "answer";
public static final String PROP_CNT = "cnt";
private String answer;
private Long cnt;
public SurveyAnswerStatistics(HashMap<String, Object> values) {
this.answer = (String) values.get(PROP_ANSWER);
this.count = (Long) values.get(PROP_CNT);
}
}
#Query("SELECT v.answer as "+PROP_ANSWER+", count(v) as "+PROP_CNT+" FROM Survey v GROUP BY v.answer")
List<SurveyAnswerStatistics> findSurveyCount();
Code needs Lombok for resolving #Getter
#Repository
public interface ExpenseRepo extends JpaRepository<Expense,Long> {
List<Expense> findByCategoryId(Long categoryId);
#Query(value = "select category.name,SUM(expense.amount) from expense JOIN category ON expense.category_id=category.id GROUP BY expense.category_id",nativeQuery = true)
List<?> getAmountByCategory();
}
The above code worked for me.
I used custom DTO (interface) to map a native query to - the most flexible approach and refactoring-safe.
The problem I had with this - that surprisingly, the order of fields in the interface and the columns in the query matters. I got it working by ordering interface getters alphabetically and then ordering the columns in the query the same way.
I just solved this problem :
Class-based Projections doesn't work with query native(#Query(value = "SELECT ...", nativeQuery = true)) so I recommend to define custom DTO using interface .
Before using DTO should verify the query syntatically correct or not
Get data with column name and its values (in key-value pair) using JDBC:
/*Template class with a basic set of JDBC operations, allowing the use
of named parameters rather than traditional '?' placeholders.
This class delegates to a wrapped {#link #getJdbcOperations() JdbcTemplate}
once the substitution from named parameters to JDBC style '?' placeholders is
done at execution time. It also allows for expanding a {#link java.util.List}
of values to the appropriate number of placeholders.
The underlying {#link org.springframework.jdbc.core.JdbcTemplate} is
exposed to allow for convenient access to the traditional
{#link org.springframework.jdbc.core.JdbcTemplate} methods.*/
#Autowired
protected NamedParameterJdbcTemplate jdbc;
#GetMapping("/showDataUsingQuery/{Query}")
public List<Map<String,Object>> ShowColumNameAndValue(#PathVariable("Query")String Query) throws SQLException {
/* MapSqlParameterSource class is intended for passing in a simple Map of parameter values
to the methods of the {#link NamedParameterJdbcTemplate} class*/
MapSqlParameterSource msp = new MapSqlParameterSource();
// this query used for show column name and columnvalues....
List<Map<String,Object>> css = jdbc.queryForList(Query,msp);
return css;
}
//in Service
`
public List<DevicesPerCustomer> findDevicesPerCustomer() {
LOGGER.info(TAG_NAME + " :: inside findDevicesPerCustomer : ");
List<Object[]> list = iDeviceRegistrationRepo.findDevicesPerCustomer();
List<DevicesPerCustomer> out = new ArrayList<>();
if (list != null && !list.isEmpty()) {
DevicesPerCustomer mDevicesPerCustomer = null;
for (Object[] object : list) {
mDevicesPerCustomer = new DevicesPerCustomer();
mDevicesPerCustomer.setCustomerId(object[0].toString());
mDevicesPerCustomer.setCount(Integer.parseInt(object[1].toString()));
out.add(mDevicesPerCustomer);
}
}
return out;
}`
//In Repo
` #Query(value = "SELECT d.customerId,count(*) FROM senseer.DEVICE_REGISTRATION d where d.customerId is not null group by d.customerId", nativeQuery=true)
List<Object[]> findDevicesPerCustomer();`

How do I use Activator.CreateInstance to do the following please

I use a similar style of code many times in my application to read in records from a database
WorkoutResultsRecord is inherited from a class called BaseRecord. One of the base constructors takes a IDataReader parameter to read fields into the class (as seen below).
What I want to do is define a generic function that will do the following for any/all of my 60+ xxxRecord type classes ie I can pass in a type as a parameter and it will return the correct type of objects as a typed List. Is it possible with Activator class? I've not used it before and my results just wouldn't compile
protected List<WorkoutResultsRecord> ReadRecordList(string sql,
IDbConnection connection)
{
var results = new List<WorkoutResultsRecord>();
using (IDbCommand command = GetCommand(sql, connection))
using (IDataReader reader = command.ExecuteReader())
while (reader.Read())
results.Add(new WorkoutResultsRecord(reader));
return results;
}
My really bad, failed attempt :(
private void sfsdf(Type type)
{
List<typeof(type)> lst = new List<type>();
Activator.CreateInstance(List<typeof(type)>);
}// function
this should work:
private void sfsdf(Type type)
{
Type genericType = typeof(List<>).MakeGenericType(type);
System.Collections.IList theList = (IList) Activator.CreateInstance(genericType);
// do whatever you like with this list...
}
Note: as the type is known at runtime only, it's not possible for you to declare a List when you write the code, so rather, use IList interface instead, but the created object theList should be of the expected type...
The following is the full function with all the generics done as I wanted. This will save a lot of typing!! thanks very much
allResults = (List<WorkoutResultsRecord>)FillList(typeof(WorkoutResultsRecord),
sql, connection, new KVP("FROMDATE", fromUtf.Date),
new KVP("TODATE", endDate.AddDays(1).Date));
IList FillList(Type type,string sql,IDbConnection connection,
params KVP[] parameters)
{
Type genericType = typeof(List<>).MakeGenericType(type);
IList results = (IList)Activator.CreateInstance(genericType);
using (var command= Command(sql,connection))
{
foreach(KVP parameter in parameters)
CreateParam(command,parameter.Key,parameter.Value);
using (IDataReader reader = command.ExecuteReader())
while (reader.Read())
results.Add(Activator.CreateInstance(type,reader));
}
return results;
}

Request server using Axis2 RPC way, parameter order in xml packets not correct

For example, I will send an object Fruits to server side.
The code like this:
public static <T> T call(String url, String ns, String method, Fruits fruits, Class<T> clz) throws AxisFault
{
RPCServiceClient client = new RPCServiceClient();
Options option = client.getOptions();
EndpointReference erf = new EndpointReference(url);
option.setTo(erf);
QName name = new QName(ns, method);
Object[] object = new Object[]{fruits};
Class[] returnTypes = new Class[]{clz};
Object[] reto = client.invokeBlocking(name, object, returnTypes);
T t = (T)reto[0];
return t;
}
The object like this:
public class Fruits implements Serializable
{
private int pear;
private int banana;
private int apple;
public int setPear(int pear){this.pear=pear;}
public int getPear(){return this.pear;}
...
}
The xml part should be this:
...
<fruits>
<pear>10</pear>
<banana>20</banana>
<apple>60</apple>
</fruits>
...
But in fact like this:
...
<fruits>
<apple>60</apple>
<banana>20</banana>
<pear>10</pear>
</fruits>
...
Axis2 makes object's property alphabetical order, but the server doesn't accept. I can't modify the serverside, it is ESB.
The only way to do a success request is to use the Axis2 generated code, I used to use WSDL2Java, but too many redundant code and difficult to maintain. So I want refactor.
I have also tried to use CXF, but it also makes object's property alphabetical order, not followed with WSDL/XSD or DTO defined style.
I've find the reason why CXF makes the ordering, it uses java.beans.BeanInfo to get properties of object, such as:
...
BeanInfo beanInfo = Introspector.getBeanInfo(Fruits.class);
PropertyDescriptor[] propertyDescriptors = beanInfo.getPropertyDescriptors();
...
The property in the array has already alphabetical order.
Who knows how to let Axis2 to serialize the Fruits' property to be correct ordering.
Thank you, the first!
Not sure on Axis2, but if you are using CXF with the JAXB databinding, you can add an annotation like:
#XmlType(name = "fruits", propOrder = { "apple", "banana", "pear" }})
to the Fruits class to tell JAXB what order you need/want them output.

Resources