Add field to App Engine-hosted database - database

I'm currently developing a mobile application who uses a Google App Engine-hosted web service.
But i'm facing an issue. I just want to add a field in one my database's table.
App Engine doesn't use classic SQL syntax, but GQL. So i cannot use the ALTER TABLE statement. How can i do this with GQL ? I looked for a solution on the web, but there's not a lot of help.
public MyEntity() {
}
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Key idStation;
private String name;
private double longitude;
private double latitude;
private java.util.Date dateRefresh = new Date(); //the field i want to add in DB
So, now when i create a "MyEntity" object, it should add the "dateRefresh" field into the database... I create my object like this:
MyEntity station = new MyEntity();
station.setName("test");
station.setLatitude(0);
station.setLongitude(0);
station.setDateRefresh(new Date("01/01/1980"));
DaoFactory.getStationDao().addStation(station);
addStation method:
#Override
public MyEntity addStation(MyEntity station) {
EntityManager em = PersistenceManager.getEntityManagerFactory().createEntityManager();
try {
em.getTransaction().begin();
em.persist(station);
em.getTransaction().commit();
} finally {
if(em.getTransaction().isActive()) em.getTransaction().rollback();
em.close();
}
return station;
}
The field "dateRefresh" is never created into my DB...
Someone to help me please ?
Thanks in advance

Just add another field to your data structure, maybe providing a default clause, and that's all. For example, if you have a UserAccount:
class UserAccount(db.Model):
user = db.UserProperty()
user_id = db.StringProperty()
you may easily add:
class UserAccount(db.Model):
user = db.UserProperty()
user_id = db.StringProperty()
extra_info = db.IntegerProperty(default=0)
timezone = db.StringProperty(default="UTC")
and let it go.

While the datastore kinda mimics tables, data is stored on a per entity basis. There is no schema or table.
All you need to do is update your model class, and new entities will be saved with the structure (fields) of the new entity.
Old entities and indexes, however, are not automatically updated. They still have the same fields as they had when they were originally written to the datastore.
There's two ways to do this. One is to make sure your code can handle situations where your new properties are missing, ie make sure no exceptions are thrown, or handle the exceptions properly when you're missing the properties.
The second way is to write a little function (usu a mapreduce function) to update every entity with appropriate or null values for your new properties.
Note that indexes are not updated unless the entity is written. So if you add a new indexed property, old entities won't show up when you query for the new property. In this case, you must use the second method and update all the entities in the datastore so that they are indexed.

Related

How do you search on String fields in google appengine datastore

I have a google appengine project that where I am storing users.
public void createUser(String loginId, String password) {
Entity e = new Entity("User");
e.setProperty("loginId", loginId);
e.setProperty("password", specialPasswordEncryptor(password));
ds.put(e);
}
Later I want to search for the User Entity by loginId when someone tries to login. But the problem is String properties are not indexed.
In the past I used add filter like this and it worked:
public Entity findUserByLoginId(String loginId) {
Query query = new Query("User").addFilter("loginId",Query.FilterOperator.EQUAL, loginId);
Entity eUser = datastore.prepare(query).asSingleEntity();
return eUser;
}
However addFilter is deprecated. I've tried other ways of doing this but keep running into the problem that String properties are not indexed so I cannot search on loginId.
Can someone point me in the right direction on this?
Thanks
String properties certainly can be indexed, unless they're longer than 1500 characters (javadoc).
To filter, you want to use setFilter(), like this:
Query query = new Query("User").setFilter(
new FilterPredicate("loginId", FilterOperator.EQUAL, loginId));

JDO and Cloud SQL update objects with makePersistentAll

I am using google cloud SQL with JDO. When I try to use the JDO PersistenceManager to store new objects with a new key it works fine, however I get an error when I try to update entities already inserted in the db:
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '503062001-43661003' for key 'PRIMARY'
The key is indeed duplicated, but I want to update that object.
Is it possible to do this with the PersistentManager.makePersistentAll() method or in an another way that avoids to write the UPDATE query manually?
More details:
The object I am trying to persist is defined like this:
PersistenceCapable(table = "xxx")
public class XXX {
#PrimaryKey
#Index(name = "xxx_id")
private Long userId;
#PrimaryKey
#Index(name = "xxx_idx")
#Column(length = 128)
private String otherId;
...
}
If you want to update the object then you retrieve the object first, and update it (either in the same txn, or detach it and update it whilst detached). All of that would be described in the JDO spec
For example https://db.apache.org/jdo/pm.html and page down to "Update Objects"

Add a new attribute to entity in datastore?

I have an entity in my app engine datastore. There's actually only one instance of this entity. I can see it in my admin console. Is it possible to add a new attribute to the entity via the admin console (using gql perhaps)?
Right now it looks something like:
Entity: Foo
Attributes: mName, mAge, mScore
and I'd like to add a new boolean attribute to this entity like "mGraduated" or something like that.
In the worst case I can write some code to delete the entity then save a new one, but yeah was just wondering.
Thanks
-------- Update ---------
Tried adding the new attribute to my class (using java) and upon loading from the datastore I get the following:
java.lang.NullPointerException:
Datastore entity with kind Foo and key Foo(\"Foo\") has a null property named mGraduated.
This property is mapped to com.me.types.Foo.mGraduated, which cannot accept null values.
This is what my entity class looks like, I just added the new attribute (mGraduated), then deployed, then tried loading the single entity from the datastore (which produced the above exception):
#PersistenceCapable
public class Foo
{
#PrimaryKey
private String k;
/** Some old attributes, look like the following. */
#Persistent
#Extension(vendorName = "datanucleus", key = "gae.unindexed", value="true")
private String mName;
...
/** Tried adding the new one. */
#Persistent
#Extension(vendorName = "datanucleus", key = "gae.unindexed", value="true")
private boolean mGraduated;
The only way to implement this is to use Boolean as the type for the new property..
Than in set method you can accept boolean value, that's no issue.
If you want the get method to also return boolean.. you also can, but be sure to check if the value is null and if so.. return default value (e.g. true)
so
private Boolean newProp = null; // can also assing default value .. e.g. true;
public void setNewProp(boolean val)
{
this.newProp = val;
}
public boolean getNewProp()
{
if(this.newProp == null)
return true; // Default value if not set
return this.newProp.booleanValue();
}
I recommend you not to migrate your data in this case - it can be very costly and can deplete your quota easily (read old data, create new, delete old = 3 operations for every entry in you data store)
You can't do this through the admin console, but you shouldn't have to delete the entity. Instead just update it- the Datastore does not enforce schemas for Kinds.
E.g., if Foo is a subclass of db.Model (Python), change your model subclass to include the new property; fetch the model instance (e.g., by its key), update the instance, including setting the value of the new field; and save the modified instance. Since you just have one instance this is easy. With many such instances to update you'd probably want to do this via task queue tasks or via a mapreduce job.
You have declared the new mGraduated field using the primitive type boolean, which cannot be null. The existing entity can't be loaded into the model class because it doesn't have this property. One option is to declare this property using the Boolean class, which can accept a null value.
The Admin Console only knows about properties in existing entities. You cannot use the Admin Console directly to create a new property with a name not used by any existing entities. (This is just a limitation of the Console. App code can do this easily.)

Playframework Siena Filtering and Ordering

This is my first question on any of these websites so pardon my unprofessionalism.
I use playframework with SIENA module (with GAE) and I came accross the following problem:
Given 3 entities:
public class Meeting extends Model{
#Id
public Long id;
public String place;
#Owned
Many<MeetingUser> users;
.
.
.
}
public class User extends Model{
#Id
public Long id;
public String firstName;
public String lastName;
#Owned
Many<MeetingUser> meetings;
.
.
.
}
public class MeetingUser extends Model{
#Id
public Long id;
public Meeting meeting;
public User user;
.
.
.
public User getUser(){
return Model.all(User.class).filter("id", user).get();
}
public Meeting getMeeting(){
return Model.all(Meeting.class).filter("id", meeting).get();
}
}
For instance I am listing a meeting and all their users:
public static void meetingInfo(Long meetingId){
Meeting meeting = Models.all(Meeting.class).filter("id",meetingId);
List<MeetingUser> meetingusers = meeting.asList();
List<User> users = new ArrayList<User>();
for(MeetingUser mu: meetingusers){
users.add(mu.getUser());
}
render(users);
}
This is done(is there any better way here?) however when it comes to filtering (especially dynamic filtering for many many fields) I can not use the Query's filter method on the MeetingUser as I need to filter on a MeetingUser's field's field (firstName). The same problem arise for ordering. I need the solution for both problems.
I hope my problem is clear and I appreciate any kind of help here.
Remember that you are in GAE which is a NoSQL DB.
So you can't do Join request as in RDBMS.
Yet, this is not really the pb you have so this was just to be sure you are aware of it ;)
So if you want to find the person having given firstname in a given meeting, can you try the following:
List<MeetingUser> meetingusers = meeting.users.asQuery().filter("firstname", "XXX");
(you can also order)
Nevertheless, knowing that you can't join, remember that you can't write a query searching for a meeting in which there are users whose firstname is XXX as it would require some joins and it doesn't exist in GAE. In this case, you need to change your model following NoSQL philosophy but this is another subject
regards
Let's try to give a way to do what you want...
Your relation is a Many-to-Many which is always the worst case :)
You want to filter Meeting by User's firstname.
It requires a join request which is not possible in GAE. In this case, you must change your model by denormalizing it (sometimes use redundancy also) and manage the join by yourself. Actually, you must do the job of the RDBMS by yourself. It seems overkill but in fact, it's quite easy. The only drawback is that you must perform several requests to the DB. NoSQL means No Schema (& No Join) so there are a few drawbacks but it allows to scale and to manage huge data load... it depends on your needs :)
The choice you did to create the MeetingUser which is a "joined" table and a kind of denormalization is good in GAE because it allows to manage the join yourself.
Solution:
// fetch users by firstname
List<User> users = users.all().filter("firstName", "John").fetch();
// fetch meetingusers associated to these users (verify the "IN" operator works because I didn't use that for a long time and don't remember if it works with this syntax)
List<MeetingUser> meetingusers = MeetingUser.all().filter("user IN", users);
// now you must fetch the whole meeting because in MeetingUser, only the Meeting ID is stored (other fields are Null or O)
List<Meeting> meetings = new ArrayList<Meeting>()
for(MeetingUsers mu:meetingusers) {
meetings.add(meetingusers.meeting);
}
// use the batch feature to fetch all objects
Meeting.batch(Meeting.class).get(meetings);
// you have your meetings
Hope this helps!

Need advice on complex insert scenario using LinqToSql with repository pattern

I have a User table and a ClubMember table in my database. There is a one-to-one mapping between users and club members, so every time I insert a ClubMember, I need to insert a User first. This is implemented with a foreign key on ClubMember (UserId REFERENCES User (Id)).
Over in my ASP.NET MVC app, I'm using LinqToSql and the Repository Pattern to handle my persistence logic. The way I currently have this implemented, my User and ClubMember transactions are handled by separate repository classes, each of which uses its own DataContext instance.
This works fine if there are no database errors, but I'm concerned that I'll be left with orphaned User records if any ClubMember insertions fail.
To solve this, I'm considering switching to a single DataContext, which I could load up with both inserts then call DataContext.SubmitChanges() only once. The problem with this, however, is that the Id for User is not assigned until the User is inserted into the database, and I can't insert a ClubMember until I know the UserId.
Questions:
Is it possible to insert the User into the database, obtain the Id, then insert the ClubMember, all as a single transaction (which can be rolled back if anything goes wrong with any part of the transaction)? If yes, how?
If not, is my only recourse to manually delete any orphaned User records that get created? Or is there a better way?
You can use System.Transactions.TransactionScope to perform this all in an atomic transaction, but if you are using different DataContext instances, it will result in a distributed transaction, which is probably not what you really want.
By the sounds of it, you're not really implementing the repository pattern correctly. A repository should not create its own DataContext (or connection object, or anything else) - these dependencies should be passed in via a constructor or public property. If you do this, you'll have no problem sharing the DataContext:
public class UserRepository
{
private MyDataContext context;
public UserRepository(MyDataContext context)
{
if (context == null)
throw new ArgumentNullException("context");
this.context = context;
}
public void Save(User user) { ... }
}
Use the same pattern for ClubMemberRepository (or whatever you call it), and this becomes trivial:
using (MyDataContext context = new MyDataContext())
{
UserRepository userRep = new UserRepository(context);
userRep.Save(user);
ClubMemberRepository memberRep = new ClubMemberRepository(context);
memberRep.Save(member);
context.SubmitChanges();
}
Of course, even this is a little bit iffy. If you have a foreign key in your database, then you shouldn't even need two repositories, because Linq to SQL manages the relationship. The code to create should simply look like this:
using (MyDataContext context = new MyDataContext())
{
User user = new User();
user.Name = "Bob";
user.ClubMember = new ClubMember();
user.ClubMember.Club = "Studio 54";
UserRepository userRep = new UserRepository(context);
userRep.Save(user);
context.SubmitChanges();
}
Don't fiddle with multiple repositories - let Linq to SQL handle the relationship for you, that's what ORMs are for.
Yes, you can do it in a single transaction. Use the TransactionScope object to begin and commit the transaction (and rollback if there is an error of course)

Resources