After some refactoring, we are having issues with the objectify queries that we are using in the application. The strange thing is that even if we revert to the original code the problem stays.
When the application starts, a number of 250 books are fetched from the Datastore using Objectify. The caching is enabled and seems to be working.
The problem is that it takes around 50 - 60 seconds to get the result, and for this reason sometimes the http request is killed. We never had this issues before and we can't find an answer to it.
If I ran a query like "select * from BookEntity order by creationDate desc limit 250" in the Google Datastore console and it took 5 - 7 seconds not more.
Before the refactoring, the book entity looked something like this:
#Index
#Entity
#Cache
public class BookEntity {
#Index
public String title_name;
#Index
public String author_name;
public String isbn;
public int number_of_pages;
public Ref<PdfEntity> book_pdf;
}
Now it's like this:
#Index
#Entity
#Cache
public class BookEntity {
#Index
#AlsoLoad("title_name")
private String titleName;
#Index
#AlsoLoad("author_name")
private String authorName;
private String isbn;
#AlsoLoad("number_of_pages")
private int numberOfPages;
#AlsoLoad("book_pdf")
private Ref<PdfEntity> bookPdf;
// getters and setters for the fields because now they are private
}
Here is just an example, but in reality it has around 20 fields.
In order to migrate the schema to the field names, I ran a task in GAE which loaded and then saved again all the BookEntity entities.
This example can be extended to all the entities that are used in the application, but the book is the worst performing one. Even though nothing is changed in the query, and we are talking about a basic query which fetches the newest 250 books by creationDate, it takes a lifetime to get the actual result. Any idea how I can investigate this issues further?
Problem found. We were persisting some information in the non-args constructor of the BookEntity, so for every book fetched from the datastore 3 save operations were made for some other entities which are referred from the book.
Related
I am working on a Spring-MVC using Postgres application in which I am trying to do a report generation form. Now, for this, I have to save the data for the form. But, the report has this matrix kind of part, which I don't know how to realize. Sure I can do it, but I want something optimized.
As you can see from the image, on left side, there are fields and each field has different values to be inserted as indicated.
As of now, I was able to come up only one Table as Parts and its class is mentioned below. But as each variable in the class will have 6 values, it will require me to create 6 tables and have some mapping. I want to avoid that. What can I do?
#Entity
#Table(name = "containment")
public class Containment {
#Id
#Column(name="containment_id")
#GeneratedValue(strategy = GenerationType.SEQUENCE,generator = "containment_gen")
#SequenceGenerator(name = "containment_gen",sequenceName = "containment_seq")
private Long containmentId;
#Column(name = "parts_at_plant")
private String partsAtPlant;
#Column(name = "parts_at_logistics")
private String partsAtLogistics;
}
I am creating class, not writing database-tables directly. If someone wants to see above in SQL code, I am more than happy to write it. Thank you.
I am just getting started with Objectify 4 and have defined this class:
#Entity
#Cache
public class Project {
#Id public long id;
#Index public String name;
}
I have saved three such entities and would like to list them sorted by name.
What I observe is the following: if I iterate over ofy().load().type(Project.class).list() I get all three (but unsorted, obviously), however if I iterate over ofy().load().type(Project.class).order("name").list() I get only one (apparently the middle one).
What is going on here? How can I list all entities sorted by name (without a filter)? I've noticed that another example uses parent: is this a requirement for this kind of sorting?
Most likely you saved the other two entities before you added the #Index annotation. If you want the indexes to be updated, re-save the entities.
I'm currently building an app for deployment to GAE, using Objectify 3.1. I am getting strange results when attempting to do a query with an order() clause.
My domain:
public class InvoiceLineItem
{
private int units;
private BigDecimal unitCost;
private BigDecimal extendedCost;
private String description;
#Parent Key<Invoice> invoice;
}
I am attempting to gather all of the InvoiceLineItems associated with a given Invoice using the following:
ofy ().query (InvoiceLineItem.class).ancestor (invoiceKey).list ( );
In my test case, this works just fine, returning 2 rows as expected.
However, when I try to add a sort order to the above query, like so:
ofy ().query (InvoiceLineItem.class).ancestor (invoiceKey).order ("+description").list ();
I always get 0 results. I've tried changing the order direction, the field its ordering by, the location of the order () clause in the query, all to no effect. Can anyone see anything that I'm doing wrong here?
Thanks...
There are a couple potential issues here:
The description field must be indexed
The description field must be less than 500 chars, because over 500 chars gets converted to a Text which is not indexable
Get rid of the +. It's either .order("description") or .order("-description").
I have following Objectify entity to store data in Google DataStore.
public class Record implements Serializable {
private static final long serialVersionUID = 201203171843L;
#Id
private Long id;
private String name; // John Smith
private Integer age; // 43
private String gender; // Male/Female
private String eventName; // Name of the marathon/event
private String eventCityName; // City of the event
private String eventStateName; // State of the event
private Date eventDate; // event date
//Getters & Setters
}
Now, my question is how can I query my database to get count of Records for a given eventName or event City+State? Or get a list of all City+Name.
On App Engine counting is very expensive: you basically need to query with certain condition (eventName = something), then count all results. The cost is a key-only query (1 read + 1 small operation) and increases with number of entities counted. For example counting 1 million entities would cost $0.8.
What is normally done is to keep count of things as a property inside a dedicated entity: increase the property value when count goes up (entity added) and decrease when it goes down (entity deleted).
If you plan to do this on a larger scale then understand there is a write/update limitation of about 5 writes/s per entity (entity group actually). See sharded counters for how to work around this.
I have a rather large class stored in the datastore, for example a User class with lots of fields (I'm using java, omitting all decorations below example for clarity):
#PersistenceCapable
class User {
private String username;
private String city;
private String state;
private String country;
private String favColor;
}
For some user queries, I only need the favColor property, but right now I'm doing this:
SELECT FROM " + User.class.getName() + " WHERE username == 'bob'
which should deserialize all of the entity properties. Is it possible to do something instead like:
SELECT username, favColor FROM " + User.class.getName() + " WHERE username == 'bob'
and then in this case, all of the returned User instances will only spend time deserializing the username and favColor properties, and not the city/state/country properties? If so, then I suppose all the other properties will be null (in the case of objects) or 0 for int/long/float?
Thank you
No, this isn't possible with the App Engine datastore; your entire entity is stored in a protocol buffer and must be deserialized together.
If you have really large properties that aren't often needed, it's probably a good idea to put these in a separate model, although if you're really talking about just 3 strings it's almost certainly not worth the effort.