How to use dynamic schema in spring data with mongodb? - spring-data-mongodb

Mongodb is a no-schema document database, but in spring data, it's necessary to define entity class and repository class, like following:
Entity class:
#Document(collection = "users")
public class User implements UserDetails {
#Id private String userId;
#NotNull #Indexed(unique = true) private String username;
#NotNull private String password;
#NotNull private String name;
#NotNull private String email;
}
Repository class:
public interface UserRepository extends MongoRepository<User, String> {
User findByUsername(String username);
}
Is there anyway to use map not class in spring data mongodb so that the server can accept any dynamic JSON data then store it in BSON without any pre-class define?

First, a few insightful links about schemaless data:
what does “schemaless” even mean anyway?
“schemaless” doesn't mean “schemafree”
Second... one may wonder if Spring, or Java, is the right solution for your problem - why not a more dynamic tool, such a Ruby, Python or the Mongoshell?
That being said, let's focus on the technical issue.
If your goal is only to store random data, you could basically just define your own controller and use the MongoDB Java Driver directly.
If you really insist on having no predefined schema for your domain object class, use this:
#Document(collection = "users")
public class User implements UserDetails {
#Id
private String id;
private Map<String, Object> schemalessData;
// getters/setters omitted
}
Basically it gives you a container in which you can put whatever you want, but watch out for serialization/deserialization issues (this may become tricky if you had ObjectIds and DBRefs in your nested document). Also, updating data may become nasty if your data hierarchy becomes too complex.
Still, at some point, you'll realize your data indeed has a schema that can be pinpointed and put into well-defined POJOs.
Update
A late update since people still happen to read this post in 2020: the Jackson annotations JsonAnyGetter and JsonAnySetter let you hide the root of the schemaless-data container so your unknown fields can be sent as top-level fields in your payload. They will still be stored nested in your MongoDB document, but will appear as top-level fields when the ressource is requested through Spring.
#Document(collection = "users")
public class User implements UserDetails {
#Id
private String id;
// add all other expected fields (getters/setters omitted)
private String foo;
private String bar;
// a container for all unexpected fields
private Map<String, Object> schemalessData;
#JsonAnySetter
public void add(String key, Object value) {
if (null == schemalessData) {
schemalessData = new HashMap<>();
}
schemalessData.put(key, value);
}
#JsonAnyGetter
public Map<String, Object> get() {
return schemalessData;
}
// getters/setters omitted
}

Related

Get spring-data-mongodb to honor getter/setter without backing field?

I have a general-purpose POJO:
public class Thing {
private String name;
private String etc;
public String getName() {
return name;
}
// other getters and setters
}
I'm using Spring 4.3.9 and Spring-data-mongodb 1.10.4. I want to store instances of this POJO in Mongodb, but I have some constraints:
I can't add Spring annotations to the base class (but I can subclass Thing and annotate that).
I want to use the name field as the Mongodb unique ID (mainly to avoid creating a separate unique index for it).
I want to (redundantly) store the name field as an actual field named "name", so that other consumers of the collection don't have to know that "name" is stored in the _id.
I started out trying this:
public class SpringThing extends Thing {
#Id
#Override
public String getName() {
return super.getName();
}
#Override
public void setName(String name) {
super.setName(name);
}
}
This causes spring to use the value of name for _id, but of course it doesn't store a field named "name" in Mongodb. The documentation says that spring will use a "property or field" named "id" or annotated with #Id. So I tried defining a redundant getter/setter which accesses the name field:
public class SpringThing extends Thing {
#Id
public String getId() {
return super.getName();
}
public void setId(String id) {
super.setName(id);
}
}
Unfortunately, spring ignores getId and setId here, and stores the object with an autogenerated ID. I also tried creating redundant getters/setters annotated with #Field("name"), but spring seems to ignore any getter/setter pair without an actual field.
Adding an actual ID field and storing a copy of the name there does work:
public class SpringThing extends Thing {
#Id
private String id;
#Override
public void setName(String id) {
this.id = id;
super.setName(id);
}
}
But it requires defining a pointless field named "id".
Is there a more reasonable way to get what I want? Is what I'm trying to do reasonable to begin with?
Thanks to a hint by #mp911de, I ended up creating a subclass of Thing that looks like this:
#TypeAlias("thing")
#Document(collection = "things")
public class SpringThing extends Thing {
#Id
#AccessType(Type.PROPERTY)
#JsonIgnore
public String getId() {
return super.getName();
}
public void setId(String taskName) {
super.setName(taskName);
}
}
The #TypeAlias annotation overrides the name which spring would use for the type, to cover up the fact that I've created a subclass just to add annotations.
#Id says that this is the getter for _id.
#AccessType says to access this field through the getter and setter rather than by directly accessing the field. This is what I needed; without it, spring looks for a private member variable named something like id.
#JsonIgnore is the Jackson (JSON library that we're using) annotation to prevent including the id field when serializing these objects to JSON.

Strings sometimes not retrieved with Cloud Datastore query (GAE, JDO, Endpoints, Java)

I have a problem with retrieving objects from Google Cloud Datastore using JDO. It is incredibly frustrating, because 99.5% of the time my code works perfectly, but 0.5% of the time some of the data is missing, and I can't find consistent steps to replicate the bug. I'm fairly certain that my issue is with either how I've set up my model or how I'm querying the datastore (I have a suspicion that it may be to do with lazy loading or the default fetch group, but I'm not sure).
Before I explain what's happening it would help to understand the model.
Here is a simplified version of my model:
#PersistenceCapable
#Inheritance(customStrategy = "complete-table")
public class DataObject {
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY, defaultFetchGroup="true")
protected Key theKey;
#Persistent()
protected String name;
//...
}
#PersistenceCapable
public class Class1 extends DataObject{
#Persistent()
#Element(dependent="true")
private List<Class2> listOfClass2 = new ArrayList<Class2>();
//...
}
#PersistenceCapable
public class Class2 extends DataObject{
#Persistent()
#Element(dependent="true")
private List<Class3> listOfClass2 = new ArrayList<Class3>();
//...
}
#PersistenceCapable
public class Class3 extends DataObject{
#Persistent()
private String value;
//...
}
And here is the code used to query the data store:
public class DataManager {
public DataObject get(
User user,
Class type,
Long id) throws OAuthRequestException
{
PersistenceManager mgr = getPersistenceManager();
DataObject obj = null;
try
{
obj = mgr.getObjectById(type, id);
getAllChildren(obj);
}
finally
{
mgr.close();
}
if(obj != null)
{
return obj;
}
else
{
throw new EntityNotFoundException("Entity not found");
}
}
/**
* Returns all of the children of the given object
* and their children etc. It is intended to 'touch' every object
* in the tree to accommodate for lazy loading.
*/
private List<StoredDataObject> getAllChildren(DataObject obj)
{
//...
}
}
The problem is that very occasionally, the query will be returned with all of the 'name' fields at a given level empty. For instance, If I retrieve an object of Class1, all of the child Class2 objects will have the 'name' attribute equal to "". The data is definitely in the data store because if I run the query again they will be populated correctly. I have never seen any of the other attributes empty, only the name field. Sometimes it occurs at the Class2 level, sometimes Class3, but never Class1 (as far as I have seen).
It is my understanding that any String attributes should automatically be included in the default fetch group, but am I possibly missing an annotation that forces the 'name' attribute to be retrieved every time?
New observation: When this occurs, it will happen consistently for about 15 minutes as long as I run the same query with the same user credentials. Could this be something to do with caching?

JDO javax.jdo.JDOFatalUserException

First of all, I am kinda a noob on this. So, I am trying to build a WebApp using GWT2.6.1 and GAE1.9.9.
I've done something like this...
#PersistenceCapable
#Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE)
public abstract class Person implements IsSerializable {
#PrimaryKey
#Persistent
private String googleUserID;
#Persistent
private String name;
#Persistent
private String secondName;
#Persistent
private String surname;
#Persistent
private Boolean isActive = false; //default value
#Persistent
private String imageURL;
...
}
then,
#PersistenceCapable
#Inheritance(strategy = InheritanceStrategy.NEW_TABLE)
public abstract class User extends Person implements IsSerializable{
#Persistent
private String email;
...
}
and finally,
#PersistenceCapable
#Inheritance(strategy = InheritanceStrategy.NEW_TABLE)
public class Admin extends User implements IsSerializable, Serializable {
private static final long serialVersionUID = 1L;
#NotPersistent
public static final AccountTypes accountType = AccountTypes.Admin;
...
}
Then I am getting the following error:
javax.jdo.JDOFatalUserException: Found inheritance strategy "new-table" on epusp.pcs.os.model.person.user.Admin. This strategy is not supported in this context. Please see the documentation for information on using inheritance with JDO: http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Inheritance
I read the documentation, but I still don't understand what I am doing wrong. Can anyone give me a hint?
PS.: I know, I know, I plan to add some new attributes to Admin and User in the future. Basically what I want to do is to check if a User is registered in database using a GoogleID and then redirect him to a specifed URL based on his AccountType (it may be an Admin, SuperUser, Auditor ...). I was doing something like this:
PersistenceManager pm = PMF.get().getPersistenceManager();
Admin user = null;
try{
user = pm.getObjectById(User.class, userId);
}finally{
pm.close();
}
switch(user.getType()){
case Admin:
return "";
case Agent:
return "";
case Auditor:
return "";
case Monitor:
return "";
case SuperUser:
return "";
default:
return null;
}
Thanks for supporting!
The "new-table" inheritance strategy allows you to split the data for a single data object across multiple "tables," but since the App Engine datastore does not support joins, operating on a data object with this inheritance strategy requires a remote procedure call for each level of inheritance. This is potentially very inefficient, so the "new-table" inheritance strategy is not supported on data classes that are not at the root of their inheritance hierarchies.
Second, the "superclass-table" inheritance strategy allows you to store the data for a data object in the "table" of its superclass. Although there are no inherent inefficiencies in this strategy, it is not currently supported. We may revisit this in future releases.
Now the good news: The "subclass-table" and "complete-table" strategies work as described in the DataNucleus documentation, and you can also use "new-table" for any data object that is at the root of its inheritance hierarchy.

Which JPA annotation should I use on an AutoPopulatingList?

I have an entity called ReferenceForm which contains an AutoPopulatingList of ReferenceItems. It looks like this:
#Entity
public class ReferenceForm implements Serializable{
private static final long serialVersionUID = -5633788166190438576L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
long id;
#lob
private AutoPopulatingList<ReferenceItem> referenceItems;
}
If I add no annotation at all to the AutoPopulatingList, the field type which hibernate creates is varbinary(255). This causes string truncation errors. To work around this, I used the #lob annotation. This felt questionable at the time, but it worked fine. At this point I was just using HSQLDB.
Now the application needs to run against MSSQL. I have generated the schema using Hibernate, and referenceItems ia an image column on the ReferenceForm table. The items themselves are stored in the ReferenceItem table.
Is #lob an appropriate annotation here?.
EDIT: ReferenceItem looks like this:
#Entity
public class ReferenceItem implements Serializable {
private static final long serialVersionUID = -9077063073733429102L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
long id;
private Title title;
private String firstName;
private String surname;
private String positionHeld;
private String institutionCompany;
#Embedded
private Address address;
#Embedded
private Telephone telephone;
private String email;
private boolean existingReference;
private String fileName;
public ReferenceItem() {
}
...getters and setters
}
SECOND EDIT:
Thanks to Willome for suggesting using #OneToMany. In the end, this is what worked.
//from
#lob
private AutoPopulatingList<ReferenceItem> referenceItems;
//to
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER)
private List<ReferenceItem> referenceItems = new AutoPopulatingList<ReferenceItem>(ReferenceItem.class);
#OneToMany accurately describes the nature of the relationship
Use the interface (List) instead of the implementation when defining the field. See http://docs.jboss.org/hibernate/core/3.3/reference/en/html/collections.html
Define the CascadeType, otherwise this error appears on saving the entity: org.hibernate.TransientObjectException: object references an unsaved transient instance
Make the FetchType EAGER otherwise you cannot load the form in a different transaction: this error appears: failed to lazily initialize a collection of role: ReferenceForm.referenceItems, could not initialize proxy - no Session
You should replace your #Lob annonation with a #OneToMany and replace the AutoPopulatingList with a collection-valued field declared as an interface type (Check out the topic 6.1. Persistent collections on this link http://docs.jboss.org/hibernate/core/3.3/reference/en/html/collections.html.)
//#Lob
#OneToMany(mappedBy = "referenceForm")
private AutoPopulatingList<ReferenceItem> referenceItems; //fail AutoPopulatingList is not an interface
#OneToMany(mappedBy = "referenceForm")
private Set<ReferenceItem> referenceItems; // OK with Set/Collection/List
Thanks to Willome for suggesting using #OneToMany. In the end, this is what worked.
//from
#lob
private AutoPopulatingList<ReferenceItem> referenceItems;
//to
#OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER)
private List<ReferenceItem> referenceItems = new AutoPopulatingList<ReferenceItem>(ReferenceItem.class);
#OneToMany accurately describes the nature of the relationship
Use the interface (List) instead of the implementation when defining
the field. See
http://docs.jboss.org/hibernate/core/3.3/reference/en/html/collections.html
Define the CascadeType, otherwise this error appears on saving the
entity: org.hibernate.TransientObjectException: object references an
unsaved transient instance
Make the FetchType EAGER otherwise you
cannot load the form in a different transaction: this error appears:
failed to lazily initialize a collection of role:
ReferenceForm.referenceItems, could not initialize proxy - no Session

java.lang.IllegalStateException : Attempting Create Multiple Associations On Variables of Class

first post here, hoping someone could perhaps shed some light on an issue I've been trying to juggle...
As a part of a school project we're attempting to build a interface to display points on a map and paths on a map.
For our first sprint I managed to work out storing/retrieving items using Objectify - it went great!
Now we're trying to extend the functionality for our next spring. Having problems now trying to store an object of type MapPath (note MapPath and MapData, our two data types, both extend class Data). Brief code snippets as follows :
#Entity
public class Data extends JavaScriptObject
{
#Id
Long id;
private String name;
private String dataSet;
...getters and setters
}
#Subclass
public class MapData extends Data implements Serializable{
{
private String name;
private String address;
private String dataSet;
#Embedded
private Coordinate location;
....constructors, getters/setters
}
#Subclass
public class PathData extends Data implements Serializable{
private String name;
private String address;
private String dataSet;
#Embedded
private Coordinate[] path;
...etc
}
Now hopefully I haven't lost you yet. I have a DataService class that basically handles all transactions. I have the following unit test :
#Test
public void storeOnePath(){
PathData pd = new PathData();
pd.setName("hi");
DataService.storeSingleton(pd);
Data d = DataService.getSingleton("hi");
assertEquals(pd,d);
}
The implementation of getSingleton is as follows :
public static void storeSingleton(Data d){
Objectify obj = ObjectifyService.begin();
obj.put(d);
}
JUnit complains:
java.lang.ExceptionInInitializerError
at com.teamrawket.tests.DataTest.storeOnePath(DataTest.java:59)
...<taken out>
Caused by: java.lang.IllegalStateException: Attempting to create multiple associations on class com.teamrawket.server.MapData for name
at com.googlecode.objectify.impl.Transmog$Visitor.addRootSetter(Transmog.java:298)
at com.googlecode.objectify.impl.Transmog$Visitor.visitField(Transmog.java:231)
at com.googlecode.objectify.impl.Transmog$Visitor.visitClass(Transmog.java:134)
at com.googlecode.objectify.impl.Transmog.<init>(Transmog.java:319)
at com.googlecode.objectify.impl.ConcreteEntityMetadata.<init>(ConcreteEntityMetadata.java:75)
at com.googlecode.objectify.impl.Registrar.registerPolymorphicHierarchy(Registrar.java:128)
at com.googlecode.objectify.impl.Registrar.register(Registrar.java:62)
at com.googlecode.objectify.ObjectifyFactory.register(ObjectifyFactory.java:209)
at com.googlecode.objectify.ObjectifyService.register(ObjectifyService.java:38)
at com.teamrawket.server.DataService.<clinit>(DataService.java:20)
... 27 more
What exactly does "attempting to create multiple associations on class ... for name" imply?
Sorry for the long post and any formatting issues that may arise.
You have repeated field names in your subclasses. You should not declare 'name' and 'dataSet' in both superclasses and subclasses; remove these fields from MapData and PathData and you should be fine.
com.teamrawket.server.MapData refers to the fullPath name for your MapData file. The name at the end refers to the field String name in your MapData class. This whole exception tries to tell you that it already contains a reference for that specific fullPath.
I would say there is another object with the same fullPath already registered. It would be helpful to know where line 59 is exactly as that is where the error occured.

Resources