Objectify - loading child entity in separate thread produces inconsistent results - google-app-engine

Question

fetch a list of objects . Use multi threading to fetch children objects and do some processing. collect all results and send to UI. threads produce inconsistent results. Some of the threads throw an error while some work fine.
sample code :
entities : Parent, Child
Parent {
Ref<Child> childKey ;
public Child getChild(){
return childKey.get();
}
}
List<Parent> parents ; //get from db
ExecutorService executorService = Executors.newFixedThreadPool(20, ThreadManager.currentRequestThreadFactory());
List<Future<ParentTO>> futures = new ArrayList<>();
for (Parent parent : parents) {
Future<ParentTO> future = executorService.submit(() -> {
return ObjectifyService.run(new Work<ParentTO>() {
public ParentTO run() {
parent.getChild(); ParentTO to = new ParentTO();
return to;
}
});
});
}
Error only shows on appengine and for some threads. It works fine on local server. If i reload page i see different results every time. Without threads it works fine.
java.util.concurrent.ExecutionException: com.googlecode.objectify.LoadException: Error loading Parent("FND1clkiTUa_lWsJow-Dxwdmadma")/Child(1): null
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at com.netkiller.erp.service.ParentService.ParentsToApprovalTOs(ParentService.java:1720)
at com.netkiller.erp.service.ParentService.getSharedParentsUI(ParentService.java:2000)
at com.netkiller.common.controller.ParentController.fetchSharedParents(ParentController.java:1266)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:214)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:749)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:690)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:83)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:945)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:876)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:961)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:852)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:837)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1772)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:108)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at com.netkiller.common.filter.OpenIdFilter.doFilter(OpenIdFilter.java:859)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at com.netkiller.common.filter.ServiceFilter.doFilter(ServiceFilter.java:100)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at com.netkiller.common.filter.SecureProtocolFilter.doFilter(SecureProtocolFilter.java:35)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at com.googlecode.objectify.ObjectifyFilter.doFilter(ObjectifyFilter.java:48)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at com.google.apphosting.runtime.jetty9.ParseBlobUploadHandler.handle(ParseBlobUploadHandler.java:119)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1182)
at com.google.apphosting.runtime.jetty9.AppEngineWebAppContext.doHandle(AppEngineWebAppContext.java:183)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at com.google.apphosting.runtime.jetty9.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:293)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at com.google.apphosting.runtime.jetty9.RpcConnection.handle(RpcConnection.java:213)
at com.google.apphosting.runtime.jetty9.RpcConnector.serviceRequest(RpcConnector.java:81)
at com.google.apphosting.runtime.jetty9.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:123)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchServletRequest(JavaRuntime.java:692)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.dispatchRequest(JavaRuntime.java:655)
at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:625)
at com.google.apphosting.runtime.JavaRuntime$NullSandboxRequestRunnable.run(JavaRuntime.java:817)
at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:269)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.googlecode.objectify.LoadException: Error loading Parent("FND1clkiTUa_lWsJow-Dxwdmadma")/Child(1): null
at com.googlecode.objectify.impl.EntityMetadata.load(EntityMetadata.java:78)
at com.googlecode.objectify.impl.LoadEngine.load(LoadEngine.java:185)
at com.googlecode.objectify.impl.LoadEngine$1.nowUncached(LoadEngine.java:141)
at com.googlecode.objectify.impl.LoadEngine$1.nowUncached(LoadEngine.java:127)
at com.googlecode.objectify.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.impl.Round$1.nowUncached(Round.java:71)
at com.googlecode.objectify.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.impl.LoaderImpl.now(LoaderImpl.java:251)
at com.googlecode.objectify.impl.ref.LiveRef.get(LiveRef.java:47)
at com.netkiller.erp.domain.Parent.getChild(Parent.java:187)
at com.netkiller.common.dto.ApprovalTO.<init>(ApprovalTO.java:503)
at com.netkiller.erp.service.ParentService$2.run(ParentService.java:1708)
at com.netkiller.erp.service.ParentService$2.run(ParentService.java:1)
at com.googlecode.objectify.ObjectifyService.run(ObjectifyService.java:81)
at com.netkiller.erp.service.ParentService.lambda$8(ParentService.java:1706)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at com.google.apphosting.runtime.ApiProxyImpl$CurrentRequestThreadFactory.lambda$newThread$0(ApiProxyImpl.java:1213)
at java.security.AccessController.doPrivileged(Native Method)
at com.google.apphosting.runtime.ApiProxyImpl$CurrentRequestThreadFactory.lambda$newThread$1(ApiProxyImpl.java:1209)
at java.lang.Thread.run(Thread.java:748)
at com.google.apphosting.runtime.ApiProxyImpl$CurrentRequestThread.run(ApiProxyImpl.java:1185)
Caused by: java.util.NoSuchElementException
at java.util.ArrayDeque.removeLast(ArrayDeque.java:295)
at com.googlecode.objectify.impl.translate.LoadContext.exitContainerContext(LoadContext.java:141)
at com.googlecode.objectify.impl.translate.ClassPopulator.load(ClassPopulator.java:119)
at com.googlecode.objectify.impl.translate.ClassTranslator.loadSafe(ClassTranslator.java:122)
at com.googlecode.objectify.impl.translate.ClassTranslator.loadSafe(ClassTranslator.java:21)
at com.googlecode.objectify.impl.translate.NullSafeTranslator.load(NullSafeTranslator.java:17)
at com.googlecode.objectify.impl.EntityMetadata.load(EntityMetadata.java:74)
... 22 more
Thanks for any insight on the issue.

Answer 1

It's hard to tell exactly what's going on from the provided code, but it's easy to accidentally contaminate data when operating across multiple threads.
To reduce the chance that you're accidentally pulling data from the wrong thread's session, use ofy().load() to load everything instead of following refs. You can pass a Ref to ofy().load().ref(theRef).
If you always call ofy() then you will always get an instance correct for your thread of execution. Try that out and if you still have an issue, we can continue this discussion with whatever error message you see.