Is rxJava library compatible with Google Appengine? If so are there any limitations? The only info I have found is mention of 'partial support' on grepcode. http://grepcode.com/snapshot/repo1.maven.org/maven2/com.netflix.rxjava/rxjava-core/0.9.0
What is not supported?
You should create a child of RxJavaSchedulersHook and override its methods using your scheduler which use com.google.appengine.api.ThreadManager:
I've done it like this :
public class MyThreadSchedulersHook extends RxJavaSchedulersHook {
private final Executor executor = new ScheduledThreadPoolExecutor(10, ThreadManager.backgroundThreadFactory());
#Override
public Scheduler getComputationScheduler() {
return Schedulers.from(executor);
}
#Override
public Scheduler getIOScheduler() {
return Schedulers.from(executor);
}
#Override
public Scheduler getNewThreadScheduler() {
return Schedulers.from(executor);
}
}
Then you should register this hook. Better to do this in an ServletContextListener implementation:
public class ContextListener implements ServletContextListener {
#Override
public void contextInitialized(final ServletContextEvent servletContextEvent) {
RxJavaPlugins.getInstance().registerSchedulersHook(new RxMainThreadSchedulersHook());
}
#Override
public void contextDestroyed(final ServletContextEvent servletContextEvent) {
// App Engine does not currently invoke this method.
}
}
It works for me.
The problem is the limitation of Java Threads in Google Appengine: https://developers.google.com/appengine/docs/java/#Java_The_sandbox
RxJava uses Java Thread and Executor in the Scheduler implementations. So the codes which get involved some concurrent Schedulers can not run in Google Java Sandbox.
If you want to use Scheduler in Google Appengine, you need to implement a special Scheduler by yourself. Besides, some operators may use Scheduler by default, such as delay use Schedulers.computation() by default. Remember to use their Scheduler overload methods.
Related
I have a Java App Engine project and I am using DeferredTasks for push queues.
/** A hypothetical expensive operation we want to defer on a background task. */
public static class ExpensiveOperation implements DeferredTask {
#Override
public void run() {
System.out.println("Doing an expensive operation...");
// expensive operation to be backgrounded goes here
}
}
I want to be able to create multiple shards of a DeferredTask to be able to have more through-put. Basically, I want to run one DeferredTask that then runs many more DeferredTasks (up to 1,000 of them). Essentially a fan-out task. How can I do that?
One issue is that when creating tasks you need to specify the name of them in the queue.yaml file. But if I want to have 1,000 tasks, do I really need to specify 1,000 of them in that file? It would get very tedious to write out "task-1", "task-2", etc.
Is there a better way to do this?
This is usually done by specifying a shard parameter for each task and reusing the same queue. As noted in your example, the entire java object is serialized with DeferredTask. So you can simply pass in any values you want in a constructor. E.g.
public static class ShardedOperation implements DeferredTask {
private final int shard;
public ShardedOperation(int shard) {
this.shard = shard;
}
}
...
#Override
public void run() {
System.out.println("Fanning out an expensive operation...");
Queue queue = QueueFactory.getDefaultQueue();
for (int i = 0; i < 1000; ++i) {
queue.add(TaskOptions.Builder.withPayload(new ShardedOperation(i)));
}
}
This matches the section you linked to https://cloud.google.com/appengine/docs/standard/java/taskqueue/push/creating-tasks#using_the_instead_of_a_worker_service where the default queue is used.
I want to use a non serializable object in stream.map() like this
stream.map { i =>
val obj = new SomeUnserializableClass()
obj.doSomething(i)
}
It is very inefficient, because I create many SomeUnserializableClass instance. Actually, it can be created only once in each worker.
In Spark, I can use mapPartition to do this. But in flink stream api, I don't known.
If you are dealing with a non serializable class what I recommend you is to create a RichFunction. In your case a RichMapFunction.
A Rich operator in Flink has a open method that is executed in the taskmanager just one time as initializer.
So the trick is to make your field transient and instantiate it in your open method.
Check below example:
public class NonSerializableFieldMapFunction extends RichMapFunction {
transient SomeUnserializableClass someUnserializableClass;
#Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
this.someUnserializableClass = new SomeUnserializableClass();
}
#Override
public Object map(Object o) throws Exception {
return someUnserializableClass.doSomething(o);
}
}
Then your code will looks like:
stream.map(new NonSerializableFieldMapFunction())
P.D: I'm using java syntax, please adapt it to scala.
I'm having trouble following the second step here.
I really don't understand how this sample does anything other than return a simple toast message. How does it utilize the API to display that message?
class EndpointsAsyncTask extends AsyncTask<Pair<Context, String>, Void, String> {
private static MyApi myApiService = null;
private Context context;
#Override
protected String doInBackground(Pair<Context, String>... params) {
if(myApiService == null) { // Only do this once
MyApi.Builder builder = new MyApi.Builder(AndroidHttp.newCompatibleTransport(),
new AndroidJsonFactory(), null)
// options for running against local devappserver
// - 10.0.2.2 is localhost's IP address in Android emulator
// - turn off compression when running against local devappserver
.setRootUrl("http://10.0.2.2:8080/_ah/api/")
.setGoogleClientRequestInitializer(new GoogleClientRequestInitializer() {
#Override
public void initialize(AbstractGoogleClientRequest<?> abstractGoogleClientRequest) throws IOException {
abstractGoogleClientRequest.setDisableGZipContent(true);
}
});
// end options for devappserver
myApiService = builder.build();
}
context = params[0].first;
String name = params[0].second;
try {
return myApiService.sayHi(name).execute().getData();
} catch (IOException e) {
return e.getMessage();
}
}
#Override
protected void onPostExecute(String result) {
Toast.makeText(context, result, Toast.LENGTH_LONG).show();
}
I'm afraid my this sample is too complex for my limited knowledge. How exactly do I "talk" to the Google Endpoints Module when running an app? Specifically, What is EndpointsAsyncTask();?
Are there any resources listing all the methods available to me? Is there a simpler example of an app communicating with a Google Cloud Endpoint?
The service methods available to you are defined by the backend source in section 1.
In the example you posted, this line: myApiService.sayHi(name).execute()
is an actual invocation call to the backend that you defined by annotating #ApiMethod("sayHi") on the method in the MyEndpoint.java class of your backend module.
The reason your Android app defines an EndpointsAsyncTask is because slow operations such as calls that hit the network need to happen off of the UI thread to avoid locking the UI. The demo simply puts the returned value into a Toast but you could modify onPostExecute() to do whatever you'd like with the result.
For more info on Google Endpoints check out:
https://cloud.google.com/appengine/docs/java/endpoints/
And for info about using an Android AsyncTask look here:
http://developer.android.com/reference/android/os/AsyncTask.html
I want to configure a timerservice in Websphere Liberty Profile.
I defined an EJB as follows, however it does not fire on the Application start.
I am using
<feature>ejbLite-3.1</feature>
Note: I cannot use ejb3.2 for the current project.
#Singleton
public class TimerEvery5Sec {
private static int i = 0;
#Schedule(second="*/1", minute="*",hour="*", persistent=false)
public void doWork(){
System.out.println("timer: " + i++);
}
#PostConstruct
public void afterCons(){
System.out.println("TimerEvery5Sec intiated");
}
public String callMe(){
return "TimerEvery5Sec - called";
}
}
Does liberty profile for ejbLite3-1 support timers?
if so, what should be done to register it in the server.xml
The Liberty profile feature ejbLite-3.1 does not support timers. Non-persistent timers were not added to the 'Lite' feature set in the EJB specification until version 3.2.
I am trying to understand the differences between the EJBContainer class provided by Java EE 6 for embeddable unit testing of EJB modules and Arquillian.
Is there a good resource or can someone help me in understanding this? Is it worth to write Arquillian tests when I could test EJBs using an embeddable container?
Full disclosure: I'm contributor of Arquillian
Arquillian is a component model for integration testing.
By using the EJBContainer, you bring the runtime(the container in this case) in your tests. By bringing the runtime in your tests, you add configuration complexity to them. Arquillian is built on the opposite philosophy. Arquillian does the opposite thing, it brings your test to the runtime and thus it eliminates the testing bandgap (gap in complexity while moving from unit to integration testing).
You will realize the above mentioned difference in the below examples.
Test an EJB using Arquillian:
#RunWith(Arquillian.class)
public class ArquillianGreeterTest {
#Deployment
public static JavaArchive createTestArchive() {
return ShrinbkWrap.create("greeterTest.jar", JavaArchive.class)
.addClasses(Greeter.class)
.addAsManifestResource(EmptyAsset.INSTANCE, "beans.xml");
}
#EJB private Greeter greeter;
#Test
public void testGreeting() {
assertNotNull(greeter);
assertEquals("Welcome to the Arquillian Universe :)", greeter.greet());
}
}
The same example using EJBContainer, would look like:
public class EJBContainerGreeterTest {
private static EJBContainer ejbContainer;
private static Context ctx;
#BeforeClass
public static void setUpClass() throws Exception {
ejbContainer = EJBContainer.createEJBContainer();
ctx = ejbContainer.getContext();
}
#AfterClass
public static void tearDownClass() throws Exception {
if (ejbContainer != null) {
ejbContainer.close();
}
}
#Test
public void testGreeting() {
Greeter greeter = (Greeter)
ctx.lookup("java:global/classes/Greeter");
assertNotNull(greeter);
assertEquals("Welcome to the Arquillian Universe :)", greeter.greet()) ;
}
}
I hope this helps.
Arquillian eventually builds on top of these ideas. Think of it as an empty placeholder, a car, that has no engine.But it can accept different engines, e.g GlassFishEngine, Jboss Engine, Websphere Engine. So you use arquillian, you define the engine of your choice, you place as passengers your code to be tested, and you put the car for a ride.
I have rewritten a series of post for newbies regarding Arquillian so if you find the first sections relevant to your question. You can find it here here