Why does this get method stop returning correctly? - google-app-engine

I am trying to write an app engine application for my university. What I am trying to achieve right now, is to create a method which takes in a Course name, and returns a list of all the CourseYears (think of that as being like a link table e.g. if Maths is the course, and it has Year 1, year 2 and Year 3; MathsYear1, MathsYear2 and MathsYear3 would be the names of the CourseYears).
This is the code for the module (WARING: super dirty code below!):
#ApiMethod(name = "courseYears")
public ArrayList<CourseYear> courseYears(#Named("name") String name){
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Query.Filter keyFilter = new Query.FilterPredicate("name", Query.FilterOperator.EQUAL, name);
Query query = new Query("Course").setFilter(keyFilter);
PreparedQuery preparedQuery = datastore.prepare(query);
List<Entity> resultList = preparedQuery.asList(FetchOptions.Builder.withLimit(1));
Course course = ofy().load().type(Course.class).id(resultList.get(0).getKey().getId()).now();
ArrayList<String> courseYearNames = course.getAllCourseYearNames();
System.out.println(course.getName());
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
for(String courseYearName: courseYearNames){
Query.Filter courseNameFilter = new Query.FilterPredicate("name", Query.FilterOperator.EQUAL, courseYearName);
Query query2 = new Query("CourseYear").setFilter(courseNameFilter);
List<Entity> resL = preparedQuery.asList(FetchOptions.Builder.withLimit(1));
System.out.println("test");
CourseYear courseYear = ofy().load().type(CourseYear.class).id(resL.get(0).getKey().getId()).now();
courseYears.add(courseYear);
}
return courseYears;
}
It basically takes a Course name in, applies a filter on all courses to get the corresponding Course object, and then calls getAllCourseYearNames() on the course to get an array list containing all its CourseYears' names. (I would have loved to do this using Keys, but parameterised Objectify keys don't seem to be supported in this version of App Engine).
I then try and get the CourseYears by looping through the arraylist of names and applying the filter for each name. I print "test" each time to see how many times it is looping. Like I said, a super dirty way of doing it.
When I try passing a few course names as a parameters, it loops the correct number of times only once or twice, and after that does not loop at all (doesn't print "test"). I could understand if it never looped, but not doing it correctly once or twice and then never again. It doesn't successfully return a list of CourseYears when it does work, but rather the relevant number of NULLs - I don't know if this is relevant. I believe it successfully retrieves the course every time, as I print the name of the course after loading and it never fails to do this.
If anyone has ANY suggestions for why this may be happening, I would be incredibly grateful to hear them!
Thanks

query2 is never used in your code. You reuse preparedQuery from your previous query, which runs on a different entity kind.

Related

ndb query by KeyProperty

I'm struggling with a KeyProperty query, and can't see what's wrong.
My model is
class MyList(ndb.Model):
user = ndb.KeyProperty(indexed=True)
status = ndb.BooleanProperty(default=True)
items = ndb.StructuredProperty(MyRef, repeated=True, indexed=False)
I create an instance of MyList with the appropriate data and can run the following properly
cls = MyList
lists = cls.query().fetch()
Returns
[MyList(key=Key('MyList', 12), status=True, items=..., user=Key('User', 11))]
But it fails when I try to filter by user, i.e. finding lists where the user equals a particular entity; even when using the one I've just used for insert, or from the previous query result.
key = lists[0].user
lists = cls.query(cls.user=key).fetch()
Returns
[]
But works fine with status=True as the filter, and I can't see what's missing?
I should add it happens in a unit testing environment with the following v3_stub
self.policy = datastore_stub_util.PseudoRandomHRConsistencyPolicy(probability=0)
self.testbed.init_datastore_v3_stub(
require_indexes=True,
root_path="%s/../"%(os.path.dirname(__file__)),
consistency_policy=self.policy
)
user=Key('User', 11) is a key to a different class: User. Not MyList
Perhaps you meant:
user = ndb.KeyProperty(kind='User', indexed=True)
Your code looks fine, but I have noticed some data integrity issues when developing locally with NDB. I copied your model and code, and I also got the empty list at first, but then after a few more attempts, the data is there.
Try it a few times?
edit: possibly related?
google app engine ndb: put() and then query(), there is always one less item

System.TypeException: Cannot have more than 10 chunks in a single operation

I have this very weird error, "System.TypeException: Cannot have more than 10 chunks in a single operation", has anyone seen/encountered this before ? Please can you guide me if you know how to solve this.
I am trying to insert different types of sObjects together in a list of sObject. The list is never larger than 10 rows.
This post here:
https://developer.salesforce.com/forums/ForumsMain?id=906F000000090nUIAQ
suggests that it is not the number of different sObjects, but the order of the objects that causes this chunk limit to be exceeded. In other words, "1,1,1,2,2,2" has one chunk, the transition from "1" to "2". "1,2,3,4,5,6" has six chunks, even though the number of elements is the same. Putting the objects into the list sorted in object order is the suggested solution.
Is it possible for you to create a reasonable test case with only 2 or 3 rows?
There are two possible explanations for this issue:
As Jagular noted, you did not order the sobjects you tried to insert, so there are more than 10 'chunks' in the list.
You try to insert > 2000 records, and > 1 sobject type. This one seems like a Salesforce Bug, since the error message doesn't match the issue.
Scenario 1 and its solution
When you have a hybrid list, make sure that the objects are not scattered without any order. For example, A,B,A,B,A,B,A,B…. Salesforce has an inherent trouble in switching sObject types for more than 10 times. They call this switching limit as Chunking Limit. So, on this hybrid list, if you would have sorted it and then passed it for DML, Salesforce would have been much happier. For example. A,A,A,A,B,B,B,B… In this case, salesforce only has to switch one time (that is read all A objects –>switch –> read all B objects). The max chunk limit is 10. So, here we are safe.
listToUpdate.sort();
UPDATE listToUpdate;
Scenario 2 and its solution
Another point that we have to bear in our mind is that if the hybrid list contains more number of objects for one type, we can run into TypeException. As mentioned in the screenshot, if list contains 1001 objects of type A and 1001 objects of type B, then total objects is equal to 2002. The maximum chunks allowed is 10. So, if you do a simple math, the number of objects in each chunk would be 2002/10 = 200. Salesforce also enforces another governor limit that each chunk should not contain 200 or more than 200 objects. In this case, we will have to foresee how much objects are possible to enter this code and we have to write code to pass lists of safe size for DML every time.
Scenario 3 and its solution
Scenario 3 and its solution
Third scenario that can happen is if the hybrid list contains objects of more than 10 types, then even if the size of the list if very small, switching happens when salesforce reads different sObject. So, we have to make sure that in this case, we allot separate lists for each sObject type and then pass it on for DML. Doing this in an apex trigger or apex class would cause you some trouble as multiple DML’s are initiated in a context. Passing this kind of multiple sObject lists for DML operations in a different context would really ease up the load you pump into the platform. Consider doing this kind of logic in a Batch Apex Job rather than a apex trigger or apex class.
Hope this helps.
Below a code that should cover all 3 Scenario from Arpit Sethi.
It's a piece of code I took from this topic: https://developer.salesforce.com/forums/?id=906F000000090nUIAQ.
and modified to cover Scenario 2.
private static void saveSobjectSet(List <Sobject> listToUpdate) {
Integer SFDC_CHUNK_LIMIT = 10;
// Developed this part due to System.TypeException: Cannot have more than 10 chunks in a single operation
Map<String, List<Sobject>> sortedMapPerObjectType = new Map<String, List<Sobject>>();
Map<String, Integer> numberOf200ChunkPerObject = new Map<String, Integer>();
for (Sobject obj : listToUpdate) {
String objTypeREAL = String.valueOf(obj.getSObjectType());
if (! numberOf200ChunkPerObject.containsKey(objTypeREAL)){
numberOf200ChunkPerObject.put(objTypeREAL, 1);
}
// Number of 200 chunk for a given Object
Integer numnberOf200Record = numberOf200ChunkPerObject.get(objTypeREAL);
// Object type + number of 200 records chunk
String objTypeCURRENT = String.valueOf(obj.getSObjectType()) + String.valueOf(numnberOf200Record);
// CurrentList
List<sObject> currentList = sortedMapPerObjectType.get(objTypeCURRENT);
if (currentList == null || currentList.size() > 199) {
if(currentList != null && currentList.size() > 199){
numberOf200ChunkPerObject.put(objTypeREAL, numnberOf200Record + 1);
objTypeCURRENT = String.valueOf(obj.getSObjectType()) + String.valueOf(numnberOf200Record);
}
sortedMapPerObjectType.put(objTypeCURRENT, new List<Sobject>());
}
sortedMapPerObjectType.get(objTypeCURRENT).add(obj);
}
while(sortedMapPerObjectType.size() > 0) {
// Create a new list, which can contain a max of chunking limit, and sorted, so we don't get any errors
List<Sobject> safeListForChunking = new List<Sobject>();
List<String> keyListSobjectType = new List<String>(sortedMapPerObjectType.keySet());
for (Integer i = 0;i<SFDC_CHUNK_LIMIT && !sortedMapPerObjectType.isEmpty();i++) {
List<Sobject> listSobjectOfOneType = sortedMapPerObjectType.remove(keyListSobjectType.remove(0));
safeListForChunking.addAll(listSobjectOfOneType);
}
update safeListForChunking;
}
}
Hope it helps,
Bye
Hi i kind of deviced a simple way to sort a list of different sobject types
public List<Sobject> SortRecordsByType(List<Sobject> records){
List<Sobject> response = new List<Sobject>();
Map<string,List<Sobject>> sortDictionary = new Map<string,List<Sobject>>();
for(Sobject record : records){
string objectTypeName = record.getSobjectType().getDescribe().getName();
if(sortDictionary.containsKey(objectTypeName)){
sortDictionary.get(objectTypeName).add(record);
}else{
sortDictionary.put(objectTypeName , new List<Sobject>{record});
}
}
// arrange in order
for(string objectName : sortDictionary.keySet()){
response.addAll(sortDictionary.get(objectName));
}
return response;
}
hopefully this solves your problem .

Grails/GORM domain saving - transient object workaround

I found a work around to a problem I had, and I want to know if it is valid or not. It is a similar problem to: Grails Gorm : Object references an unsaved transient instance
Lets assume I have two domain Objects (names changed to protect the guilty).
public class Shelf {
String name
Set<Book> books = [] as Set
static hasMany = [books: Book]
}
and
public class Book {
String title
Shelf shelf
}
So this means that 1 Shelf contains 0 to many Books, and one Book can be on only one Shelf.
This Shelf is very large. And at some point, it contains 80,000 Books. All stored nicely in the DB. Of course, adding new Books is getting slower and slower.
This is done by:
Book book1 = new Book("Awesome Title")
existingShelf.addToBooks(book1)
existingShelf.save(flush: true) // super slow
This is slow. Mainly (I assume) because GORM has to confirm the other 80,000 records.
So I did this to try to work around the slow point.
Book book2 = new Book("Awesome Title 2")
book2.save(flush: true)
This gives me an "Object references an unsaved transient instance", which I guess makes sense - the "shelf" value is empty.
So I did something a little weird:
Book book3 = new Book("Awesome Title 3")
book3.shelf = new Shelf()
book3.shelf.id = <known/valid id here>
book2.save(flush: true)
This works. It saves. There are no referential errors. Further code that depends on this... works.
I just made a call that last minutes and reduced it down to seconds.
But that seems too easy. I'm sure I worked around Grails magic some how. And probably broke something in the process.
Advice? Explanations?
Yes, using addTo* methods can be slow. If you look at the generated SQL you'll understand why. Doing the following:
new Book(title: "GORM Performance", shelf: grailsShelf).save()
will be faster and there is technically nothing wrong with it. Just be aware of that your instance of grailsShelf.books will not contain the new book until you've refreshed the collection from the database. This is part of what the addTo* method does for you.
Side note:
Set<Book> books = [] as Set
is unnecessary.

How do I SubmitChanges on multiple tables that are related in LinqToSql?

I'm making a Windows Phone 7.1 application, and I'm having a lot of trouble submitting changes to my database. Here is the structure of the tables in my database:
Day <-1-----*-> TrainingSession <-many-----1-> Sport
So, a single day can have many training sessions, and a training session has one sport. A single sport can naturally be in many different training sessions.
The primary keys look like this:
Day - DateTime
TrainingSession - int (DB generated)
Sport - nvarchar(200)
Sports will simply have attributes sportName, and an iconFileName.
I've set up Associations by putting EntitySet in both Day and Sport, and TrainingSession has EntityRef and EntityRef. I'm not 100% sure if Sport needs the EntitySet, so please correct me if I'm wrong. For the moment, I just hard-coded some sports in my Sport class for testing, and you'll see me retrieving an ObservableCollection to get those out.
Here is how I am trying to create a collection of days with training sessions, each training session having different sports:
public void CreateDay(DateTime date)
{
FitPlanDataContext calendarDatabase = new FitPlanDataContext(FitPlanDataContext.ConnectionString);
DateTime firstDate = new DateTime(date.Year, date.Month, 1);
DayItem dayItem = new DayItem();
dayItem.DateTime = firstDate;
fillTestDayItemWithRandomData(dayItem);
calendarDatabase.DayItems.InsertOnSubmit(dayItem);
calendarDatabase.SubmitChanges();
}
private void fillTestDayItemWithRandomData(DayItem dayItem)
{
ObservableCollection<SportArt> sportArtCollection = SportArtController.GetAllSports();
dayItem.TrainingSessions = new EntitySet<TrainingSession>();
ObservableCollection<TrainingSession> trainingSessionCollection = new ObservableCollection<TrainingSession>();
TrainingSession trainingSession1 = new TrainingSession();
trainingSession1.DayItem = dayItem;
trainingSession1.SportArt = sportArtCollection[1];
trainingSessionCollection.Add(trainingSession1);
TrainingSession trainingSession2 = new TrainingSession();
trainingSession2.DayItem = dayItem;
trainingSession2.SportArt = sportArtCollection[2];
trainingSessionCollection.Add(trainingSession2);
FitPlanDataContext calendarDatabase = new FitPlanDataContext(FitPlanDataContext.ConnectionString);
calendarDatabase.TrainingSessions.InsertAllOnSubmit<TrainingSession>(trainingSessionCollection);
}
This code is not working for me, and it is giving me the following error:
NotSupportedException was Unhandled:
An attempt has been made to Attach or Add an entity that is not new, perhaps having been loaded from another DataContext. This is not supported.
Before I got this error, I was also getting NullReferenceExceptions.
I've been looking around for a solution, and I saw some people used Detach or workarounds with Attach, but I havent figured out how I could implement it to my code. Could anyone give me a helping hand with this?
Also, I thought the NullReferenceException could be coming from the fact that I'm not saving any sports to the database, could this be so?
So I messed around with it a lot, and today I finally found the solution I was looking for.
It seems I asked the question wrong. I didn't include the query from the database, which is probably important to add. I actually omitted a lot of the code to keep things simple in my question, but looks like I omitted too much.
Anyways, it turned out the way I setup the database structure was correct, and nothing had to be changed there.
So here's what I did to get it working:
-The call to the method that fills the day with training sessions needed to go after submitting changes about the day. This is because days have training sessions, and I cant save training sessions without the day already in the database.
-I added using statements around the places where I need to use the datacontext instead of just creating an instance of the datacontext with a local variable. This ensures that the datacontext lives only in the scope of the using statment.
(I changed the DateTime of the day to be the date given as the parameter to the method)
public void CreateDay(DateTime date)
{
DayItem dayItem = new DayItem();
dayItem.DateTime = date;
using (FitPlanDataContext calendarDatabase = new FitPlanDataContext(FitPlanDataContext.ConnectionString))
{
calendarDatabase.DayItems.InsertOnSubmit(dayItem);
calendarDatabase.SubmitChanges();
}
fillTestDayItemWithRandomData(dayItem);
}
Then, the changes to the method that fills the day with training sessions go like this:
-I open a using statement where I instantiate a new datacontext. Then I access the database to retrieve a list of all the sports, and also the day that I need to update. I find the day I need to update by dayItemParameter. (Remember that retrieving from the database will give you a collection.)
-I create my new training sessions and fill their properties. Note that the day I retrieved from the database is the value of a training session's property because the training session is a child of day, and needs to know who its parent day is.
-I removed the instantiation of EntitySet because I realized that I already instantiate it in the constructor of the DayItem class.
-Lastly, I add all the new training sessions into a collection, and save them all to the database at once using InsertAllOnSubmit(collection).
private void fillTestDayItemWithRandomData(DayItem dayItemParameter)
{
using (FitPlanDataContext calendarDatabase = new FitPlanDataContext(FitPlanDataContext.ConnectionString))
{
ObservableCollection<SportArt> sportArtCollection;
var sportArts = (from SportArt sportArt in calendarDatabase.SportArts
select sportArt);
sportArtCollection = new ObservableCollection<SportArt>(sportArts);
ObservableCollection<DayItem> dayItemCollection;
var dayItems = (from DayItem dayItem in calendarDatabase.DayItems
where dayItem.DateTime == dayItemParameter.DateTime
select dayItem);
dayItemCollection = new ObservableCollection<DayItem>(dayItems);
DayItem foundDayItem = dayItemCollection[0];
ObservableCollection<TrainingSession> trainingSessionCollection = new ObservableCollection<TrainingSession>();
TrainingSession trainingSession1 = new TrainingSession();
trainingSession1.DayItem = foundDayItem;
trainingSession1.SportArt = sportArtCollection[1];
trainingSessionCollection.Add(trainingSession1);
TrainingSession trainingSession2 = new TrainingSession();
trainingSession2.DayItem = foundDayItem;
trainingSession2.SportArt = sportArtCollection[2];
trainingSessionCollection.Add(trainingSession2);
calendarDatabase.TrainingSessions.InsertAllOnSubmit<TrainingSession>(trainingSessionCollection);
calendarDatabase.SubmitChanges();
}
}
Conclusion:
The main problem I was having was that I was trying to save training sessions to a day that wasn't submitted to the database. The next big problem (that I think many others have) is that reading and updating of an entity has to be in the same datacontext. So, you can't create a datacontext to retrieve a day, then use another datacontext to add a training session to that day (even if you saved the value of the day to a local variable). You need to retrieve the day and save training sessions to it all in the same data context.
At the moment, my application is working, but it is quite sluggish. In this question, I'm asking about just one day, but in my actual program, I'm creating hundreds of days, which means a lot of opening and closing of the database. If anyone has suggestions to how I can
optimize the process, I'm open ears.
I realize and apologize that this post got so long, but writing it helped me to understand the situation with more depth, and I really hope that it'll help others too.

How do I avoid STANDARD_PRICE_NOT_DEFINED when unit-testing an OpportunityLineItem in Apex v24.0?

Apparently a new feature of the Spring '12 / v24.0 release of Apex in Salesforce.com is that unit tests no longer have access to 'real' data -- thus (if I'm understanding the change correctly) a SOQL query will now only retrieve objects that have been inserted during the course of the unit test -- and even that is subject to some limitations.
At any rate this seems to throw OpportunityLineItem testing out the window, because:
It's impossible to insert an OpportunityLineItem without a PriceBookEntryId, BUT
You can't insert a new price-book entry for product X unless you already have a Standard Price Book entry for product X, BUT
There isn't a Standard Price Book in the test data because the Pricebook2 table, like all tables, is effectively empty at the beginning of the unit-test run, AND
There's no way to create a Standard Price Book in Apex
I'm really hoping I got at least one of those four points wrong, but so far no variation on my existing unit-tests has shown any of them to be wrong. Which of course means my unit tests no longer work. This happened literally overnight -- the tests ran fine in my sandbox on Friday, and now they fail.
Am I missing something, or is this a bug in the new SFDC release?
There is new functionality introduced in Summer 14, you can now use Test.getStandardPricebookId() to get the standard pricebook ID without having to set SeeAllData to True.
Firstly, to put your mind at ease, there are no plans ever to deprecate the seeAllData flag. We're not going to pull the rug out from under you. As to the creation of standard price book in an apex test, I'm not sure. There are, I'm sure, several areas where testing without existing data is difficult on the platform today, which is one reason why the seeAllData flag is there. We'll be trying to close those gaps in the next few releases.
I just ran into this, and although your post is old, it's the first result on Google so I thought I'd share what I did.
My basic architecture is a test class that calls a utility class to as a way of creating test data on the fly (there are other ways, this is my habit).
Short version:
set see all data to true
make sure the standard price book is active
add a pricebook entry for the standard price book - flag as active
add a pricebook entry for you test price book - flag as active
Test class:
#isTest (seeAllData=true)
public with sharing class RMA_SelectLineItemsControllerTest {
static testmethod void testBasicObjects() {
Pricebook2 standard = [Select Id, Name, IsActive From Pricebook2 where IsStandard = true LIMIT 1];
if (!standard.isActive) {
standard.isActive = true;
update standard;
}
Pricebook2 pb = RMA_TestUtilities.createPricebook();
Product2 prod = RMA_TestUtilities.createProduct();
PricebookEntry pbe = RMA_TestUtilities.createPricebookEntry(standard,pb,prod);
}
}
The utility method look like this (only showing that around the new PBE):
public static PricebookEntry createPricebookEntry (Pricebook2 standard, Pricebook2 newPricebook, Product2 prod) {
System.debug('***** starting one');
PricebookEntry one = new PricebookEntry();
one.pricebook2Id = standard.id;
one.product2id = prod.id;
one.unitprice = 1249.0;
one.isactive = true;
insert one;
System.debug('***** one complete, ret next');
PricebookEntry ret = new PricebookEntry();
ret.pricebook2Id = newPricebook.id;
ret.product2id = prod.id;
ret.unitprice = 1250.0;
ret.isactive = true;
insert ret;
return ret;
}
Another work around would be to make your trigger be aware of being run in a test using Test.isRunningTest(), but I think this solution misses the point of best practice, which I believe is the whole point of making tests isolated from pre-existing data.
Perhaps Salesforce could make the Pricebook2.isStandard field writeable if code is running in the context of a test, or the specific Standard Price Book record should be given the same status as User and Profile??
Please let me know if anyone has used Test.getStandardPricebookId() and able to insert opportunity line item in test class. I tried this method with below mentioned code but got an error ": STANDARD_PRICE_NOT_DEFINED, No standard price defined for this product: []".
Note: I have seeAllData=false
ID standardPBID = Test.getStandardPricebookId();
PriceBook2 pb = new PriceBook2();
pb.name = 'GEW Water CMS';
pb.isActive=true;
insert pb;
Product2 prod= new Product2();
prod.name='TestProd';
prod.productcode='4568';
prod.isActive=true;
insert prod;
PricebookEntry standardPrice = new PricebookEntry(Pricebook2Id = standardPBID, Product2Id = prod.Id, UnitPrice = 10000, IsActive = true, currencyISOCode='USD' );
PriceBookEntry pbe= new PricebookEntry(pricebook2id=pb.id, product2id=prod.id,unitprice=2000, isActive=true, currencyISOCode='EUR');
insert pbe;
OpportunityLineItem oli = new OpportunityLineItem(OpportunityId = OppList[0].Id, pricebookentryid=pbe.id, UnitPrice = 100, Quantity = 1);
insert oli;

Resources