I'm currently working in project that needs Cassandra database to have search ability. We've got DataStax cluster and we want to use Spring Data to simplify database operations. However, when we made entity that got both - #Table (for cassandra) and #SolrDocument (for Solr) it happened to be broken. The only error we got is the one below. Anyone have encountered such a problem?
Caused by: org.springframework.data.mapping.PropertyReferenceException: No property findAll found for type ENTITYNAME!
I know that this is probably Spring issue, but hope to find someone who have fought this type of problem.
Greetings!
Some sample entity causing problems:
#SolrDocument(solrCoreName = "sample_entity")
#Table("sample_entity")
#Getter
#Setter
#AllArgsConstructor
#NoArgsConstructor
public final class SampleEntity {
#PrimaryKey
#Indexed(name = "id")
private UUID id;
private LocalDateTime created;
private UUID foreignId;
#Indexed(name = "name")
private String name;
private boolean someflag = true;
}
You're mixing up things - if you're using DSE Search, then it's better to perform search via CQL, by querying in the solr_query column. In your example, the #SolrDocument will force using of the Solr's HTTP API, while #Table will force use of CQL.
You can use Object Mapper from DataStax to map classes to tables, like this:
// STest.java
#Table(keyspace = "test",name = "stest")
public class STest {
#PartitionKey
private int id;
private String t;
}
// STestAccessor.java
#Accessor
public interface STestAccessor {
#Query("SELECT * FROM test.stest WHERE solr_query = :solr")
Result<STest> getViaSolr(#Param("solr") String solr);
}
// STestMain.java
MappingManager manager = new MappingManager(session);
STestAccessor sa = manager.createAccessor(STestAccessor.class);
Result<STest> rs = sa.getViaSolr("*:*");
for (STest sTest : rs) {
System.out.println("id=" + sTest.getId() + ", text=" + sTest.getT());
}
Here is the full code.
Related
Team
I'm building a spring boot application that can support multiple DBs either Cassandra, CouchDB or DynamoDB based on the configuration in application.yml.
My entity class has annotations that are specific to Cassandra and the annotations for DynamoDB are different. For eg. DynamoDB has #DynamoDBTable for Table and Cassandra has #org.springframework.data.cassandra.core.mapping.Table annotations.
The problem is that I would like to use a single entity object irrespective of the DB type because the entity is referred from multiple places in the application. What is the best design pattern to implement this?
In case of Cassandra -
package com.abc;
#Table("Cart")
public class Cart {
#PrimaryKeyColumn(ordinal = 0, type = PrimaryKeyType.PARTITIONED)
#GeneratedValue(strategy = GenerationType.AUTO)
protected String id;
#PrimaryKeyColumn(ordinal = 1, type = PrimaryKeyType.PARTITIONED)
private String userId;
#PrimaryKeyColumn(ordinal = 2, type = PrimaryKeyType.CLUSTERED, ordering = Ordering.DESCENDING)
private String skuId;
In case of DynamoDB -
#DynamoDBTable(tableName = "Cart")
public class Cart {
#DynamoDBHashKey
#DynamoDBAutoGeneratedKey
protected String id;
private String userId;
private String skuId;
Thanks
AA
I would suggest you to create an intermediary object which can act as a bridge between your application logic and database ORM.
You can create a helper function which populate those fields.
class CartDAO {
private String id;
private String userId;
private String skuId;
// Getters & Setters
}
class CartService{
CartDAO fetchFromDynamoDB(String Id)
{
// Fetch from DB
// Create CartDAO from that object
// Return CartDAO
}
CartDAO fetchFromCassandra(String Id)
{
// Fetch from DB
// Create CartDAO from that object
// Return CartDAO
}
}
Now you can use CartDAO seamlessly in your application logic.
Yes it is possible.
Option 1:
Simply put the required annotation of both MongoDB and Cassandra.
Each annotation will have there own package and definition. So provide the required definition.
Option 2:
As defined by snk01, you can use that approach as well.
Here i am assuming that you are writing the persistence layer for each database seperately.
Mongodb is a no-schema document database, but in spring data, it's necessary to define entity class and repository class, like following:
Entity class:
#Document(collection = "users")
public class User implements UserDetails {
#Id private String userId;
#NotNull #Indexed(unique = true) private String username;
#NotNull private String password;
#NotNull private String name;
#NotNull private String email;
}
Repository class:
public interface UserRepository extends MongoRepository<User, String> {
User findByUsername(String username);
}
Is there anyway to use map not class in spring data mongodb so that the server can accept any dynamic JSON data then store it in BSON without any pre-class define?
First, a few insightful links about schemaless data:
what does “schemaless” even mean anyway?
“schemaless” doesn't mean “schemafree”
Second... one may wonder if Spring, or Java, is the right solution for your problem - why not a more dynamic tool, such a Ruby, Python or the Mongoshell?
That being said, let's focus on the technical issue.
If your goal is only to store random data, you could basically just define your own controller and use the MongoDB Java Driver directly.
If you really insist on having no predefined schema for your domain object class, use this:
#Document(collection = "users")
public class User implements UserDetails {
#Id
private String id;
private Map<String, Object> schemalessData;
// getters/setters omitted
}
Basically it gives you a container in which you can put whatever you want, but watch out for serialization/deserialization issues (this may become tricky if you had ObjectIds and DBRefs in your nested document). Also, updating data may become nasty if your data hierarchy becomes too complex.
Still, at some point, you'll realize your data indeed has a schema that can be pinpointed and put into well-defined POJOs.
Update
A late update since people still happen to read this post in 2020: the Jackson annotations JsonAnyGetter and JsonAnySetter let you hide the root of the schemaless-data container so your unknown fields can be sent as top-level fields in your payload. They will still be stored nested in your MongoDB document, but will appear as top-level fields when the ressource is requested through Spring.
#Document(collection = "users")
public class User implements UserDetails {
#Id
private String id;
// add all other expected fields (getters/setters omitted)
private String foo;
private String bar;
// a container for all unexpected fields
private Map<String, Object> schemalessData;
#JsonAnySetter
public void add(String key, Object value) {
if (null == schemalessData) {
schemalessData = new HashMap<>();
}
schemalessData.put(key, value);
}
#JsonAnyGetter
public Map<String, Object> get() {
return schemalessData;
}
// getters/setters omitted
}
I am using Objectify to store data on Google App Engine's datastore. I have been trying to implement a one-to-many relationship between two classes, but by storing a list of parameterised keys. The method below works perfectly some of the time, but returns an empty array other times - does anyone know why this may be?
It will either return the correct list of CourseYears, or
{
"items": [
]
}
Here is the method:
#ApiMethod(name = "getCourseYears") #ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public ArrayList<CourseYear> getCourseYears(#Named("name") String name){
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
System.out.println(course.getName());
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseYears.add(ofy().load().type(CourseYear.class).id(courseYearKey.getId()).now());
}
return courseYears;
}
The Course class which stores many CourseYear keys
#Entity
public class Course {
#Id
#Index
private Long courseId;
private String code;
#Index
private String name;
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public List<Key<CourseYear>> getCourseYears() {
return courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void setCourseYears(List<Key<CourseYear>> courseYears) {
this.courseYears = courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void addCourseYear(Key<CourseYear> courseYearRef){
courseYears.add(courseYearRef);
}
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
List<Key<CourseYear>> courseYears = new ArrayList<Key<CourseYear>>();
...
}
I am debugging this on the debug server using the API explorer. I have found that it will generally work at the start for a few times but if I leave and return to the API and try and run it again, it will not start working again after that.
Does anyone have any idea what might be going wrong?
Many thanks.
You might want to reduce the amount of queries you send to the datastore. Try something like this:
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
List<Long> courseIds = new List<>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseIds.add(courseYearKey.getId());
}
Map<Long, Course> courses = ofy().load().type(CourseYear.class).ids(courseIds).list();
// add all courses from map to you courseYears list
I also strongly recommend a change in your data structure / entities:
In your CourseYears add a property Ref<Course> courseRef with the parent Course and make it indexed (#Index). Then query by
ofy().load().type(CourseYear.class).filter("courseRef", yourCourseRef).list();
This way you'll only require a single query.
The two most likely candidates are:
Eventual consistency behavior of the high replication datastore. Queries (ie your filter() operation) always run a little behind because indexes propagate through GAE asynchronously. See the GAE docs.
You haven't installed the ObjectifyFilter. Read the setup guide. Recent versions of Objectify throws an error if you haven't installed it, so if you're on the latest version, this isn't it.
I'm getting a ClassCastException (java.lang.Double cannot be cast to java.lang.Long) when retrieving multiple instances of a class in the datastore. The class has many values that are doubles and many that are longs. I'd like to view what is in the datastore and compare with the class properties to see if there is a mismatch. I tried the representationsOfProperty method found near the bottom of https://cloud.google.com/appengine/docs/java/datastore/metadataqueries#Java_Kind_queries, but my queries return null.
I have a class defined similar to the following:
#PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Container
{
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long containerID = null;
#Persistent
private Long extensionID = null;
#Persistent
private String homeUrl;
#Persistent
private Double containerScore;
}
I copied the code from the GAE page linked above. The only change I made was to convert 'key' to "key" since a string is requested and what is shown in the example isn't a character.
Collection<String> representationsOfProperty(DatastoreService ds,
String kind,
String property)
{
// Start with unrestricted non-keys-only property query
Query q = new Query(Entities.PROPERTY_METADATA_KIND);
// Limit to specified kind and property
q.setFilter(new FilterPredicate("__key__", Query.FilterOperator.EQUAL, Entities.createPropertyKey(kind, property)));
// Get query result
PreparedQuery pq = ds.prepare(q);
Entity propInfo = pq.asSingleEntity();
if( null == propInfo )
{
Collection<String> strs = new ArrayList<String>();
strs.add( "[ERROR: Invalid Query: " + pq.toString() + "]" );
return strs;
}
// Return collection of property representations
return (Collection<String>) propInfo.getProperty("property_representation");
}
I call that method with the following code:
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
String prop = "containerID";
Collection<String> reps = representationsOfProperty( datastore, Container.class.toString(), prop );
Unfortunately, propInfo is always null. Any ideas on what I could try? Am I doing something wrong?
I just created a sample project to reproduce your issue. JDO is saving the entities in the Datastore, but not naming the primary key property according to the member variables of the PersistenceCapable class. Check your datastore viewer and grab a random Container entity to see what's happening.
If you need to do metadata queries on the id property in this manner, you should be aware that the ID/Name property is part of the key. Check this interesting talk to learn more about how the Datastore internals work (it's based on BigTable).
First of all, I am kinda a noob on this. So, I am trying to build a WebApp using GWT2.6.1 and GAE1.9.9.
I've done something like this...
#PersistenceCapable
#Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE)
public abstract class Person implements IsSerializable {
#PrimaryKey
#Persistent
private String googleUserID;
#Persistent
private String name;
#Persistent
private String secondName;
#Persistent
private String surname;
#Persistent
private Boolean isActive = false; //default value
#Persistent
private String imageURL;
...
}
then,
#PersistenceCapable
#Inheritance(strategy = InheritanceStrategy.NEW_TABLE)
public abstract class User extends Person implements IsSerializable{
#Persistent
private String email;
...
}
and finally,
#PersistenceCapable
#Inheritance(strategy = InheritanceStrategy.NEW_TABLE)
public class Admin extends User implements IsSerializable, Serializable {
private static final long serialVersionUID = 1L;
#NotPersistent
public static final AccountTypes accountType = AccountTypes.Admin;
...
}
Then I am getting the following error:
javax.jdo.JDOFatalUserException: Found inheritance strategy "new-table" on epusp.pcs.os.model.person.user.Admin. This strategy is not supported in this context. Please see the documentation for information on using inheritance with JDO: http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Inheritance
I read the documentation, but I still don't understand what I am doing wrong. Can anyone give me a hint?
PS.: I know, I know, I plan to add some new attributes to Admin and User in the future. Basically what I want to do is to check if a User is registered in database using a GoogleID and then redirect him to a specifed URL based on his AccountType (it may be an Admin, SuperUser, Auditor ...). I was doing something like this:
PersistenceManager pm = PMF.get().getPersistenceManager();
Admin user = null;
try{
user = pm.getObjectById(User.class, userId);
}finally{
pm.close();
}
switch(user.getType()){
case Admin:
return "";
case Agent:
return "";
case Auditor:
return "";
case Monitor:
return "";
case SuperUser:
return "";
default:
return null;
}
Thanks for supporting!
The "new-table" inheritance strategy allows you to split the data for a single data object across multiple "tables," but since the App Engine datastore does not support joins, operating on a data object with this inheritance strategy requires a remote procedure call for each level of inheritance. This is potentially very inefficient, so the "new-table" inheritance strategy is not supported on data classes that are not at the root of their inheritance hierarchies.
Second, the "superclass-table" inheritance strategy allows you to store the data for a data object in the "table" of its superclass. Although there are no inherent inefficiencies in this strategy, it is not currently supported. We may revisit this in future releases.
Now the good news: The "subclass-table" and "complete-table" strategies work as described in the DataNucleus documentation, and you can also use "new-table" for any data object that is at the root of its inheritance hierarchy.