BroadcastHub doesn't wait for first materialization of Source - akka-stream

This is what Akka Streams documentation states about Source materialized from BroadcastHub Sink (link):
The resulting Source can be materialized any number of times, each materialization effectively attaching a new subscriber. If there are no subscribers attached to this hub then it will not drop any elements but instead backpressure the upstream producer until subscribers arrive.
Therefore I would expect that this:
Source<String, NotUsed> bcast =
Source.from(Arrays.asList("1", "2", "3"))
.log("before broadcasting")
.runWith(BroadcastHub.of(String.class), materializer);
bcast.log("after broadcasting")
.runWith(Sink.ignore(), materializer);
would effectively backpressure the upstream of the BroadcastHub Sink until the bcast Source is materialized at least once, but it is not the case. If I run the above, the output is:
[before broadcasting] Element: 1
[before broadcasting] Element: 2
[before broadcasting] Element: 3
[before broadcasting] Upstream finished.
[after broadcasting] Upstream finished.
which is contrary to the documentation, as the elements are effectively dropped and never broadcasted. If I limit the buffer of the Broadcast Sink like this:
Source<String, NotUsed> bcast =
Source.from(Arrays.asList("1", "2", "3"))
.log("before broadcasting")
.runWith(BroadcastHub.of(String.class, 2), materializer); //buffer smaller than number of events
then all the elements are broadcasted:
[before broadcasting] Element: 1
[before broadcasting] Element: 2
[after broadcasting] Element: 1
[after broadcasting] Element: 2
[before broadcasting] Element: 3
[before broadcasting] Upstream finished.
[after broadcasting] Element: 3
[after broadcasting] Upstream finished.
The way I see it it's either a bug or the documentation is misleading?

I am not an Akka Expert, however, I did some more tests in Scala. (and also changed from strings to integers)
By throttling the stream to make sure the second stream connects before the hubs upstream source has completed, it works as one would expect.
If the delay of 100 millis is changed to 10 millis then the second response (below) is observed.
I havent checked the source but ...
It looks like the elements do get buffered by the broadcast hub, but if the upstream source has completed when the second sink connects, it sees that the upstream finished, and completes, without receiving the buffered elements.
I'm not sure if that's a documentation issue or a bug.
val sink=BroadcastHub.sink[Int]// (2)
//10 millis
val bcast = Source(1 to (3)).throttle(2,100.millis).
log("before broadcasting").
runWith(sink)
//Thread.sleep(1000) doesnt make any difference
bcast.log("after broadcasting").runWith(Sink.ignore)
100 ms throttle Result
[DEBUG] [01/29/2020 10:48:50.168] [broadcastHubTest-akka.actor.default-dispatcher-4] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Element: 1
[DEBUG] [01/29/2020 10:48:50.168] [broadcastHubTest-akka.actor.default-dispatcher-4] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Element: 2
[DEBUG] [01/29/2020 10:48:50.172] [broadcastHubTest-akka.actor.default-dispatcher-2] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [after broadcasting] Element: 1
[DEBUG] [01/29/2020 10:48:50.172] [broadcastHubTest-akka.actor.default-dispatcher-2] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [after broadcasting] Element: 2
[DEBUG] [01/29/2020 10:48:50.233] [broadcastHubTest-akka.actor.default-dispatcher-4] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Element: 3
[DEBUG] [01/29/2020 10:48:50.233] [broadcastHubTest-akka.actor.default-dispatcher-4] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Upstream finished.
[DEBUG] [01/29/2020 10:48:50.233] [broadcastHubTest-akka.actor.default-dispatcher-3] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [after broadcasting] Element: 3
[DEBUG] [01/29/2020 10:48:50.234] [broadcastHubTest-akka.actor.default-dispatcher-3] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [after broadcasting] Upstream finished.
10 ms throttle result
[DEBUG] [01/29/2020 10:56:26.030] [main] [EventStream(akka://broadcastHubTest)] logger log1-Logging$DefaultLogger started
[DEBUG] [01/29/2020 10:56:26.031] [main] [EventStream(akka://broadcastHubTest)] Default Loggers started
[DEBUG] [01/29/2020 10:56:26.136] [broadcastHubTest-akka.actor.default-dispatcher-2] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Element: 1
[DEBUG] [01/29/2020 10:56:26.137] [broadcastHubTest-akka.actor.default-dispatcher-2] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Element: 2
[DEBUG] [01/29/2020 10:56:26.137] [broadcastHubTest-akka.actor.default-dispatcher-2] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Element: 3
[DEBUG] [01/29/2020 10:56:26.137] [broadcastHubTest-akka.actor.default-dispatcher-2] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [before broadcasting] Upstream finished.
[DEBUG] [01/29/2020 10:56:26.139] [broadcastHubTest-akka.actor.default-dispatcher-5] [akka.stream.Log(akka://broadcastHubTest/system/StreamSupervisor-0)] [after broadcasting] Upstream finished.

Related

Apache Camel - Parallel Routes Inflight Exchanges

I have a Camel context with many routes that starts every 15m with Timer Component.
These routes set some properties in exchange (Target host, Query and Current Date that I use a Processor to get date, -12 hours and transform to GMT).
After set these properties, using Direct, another route is called to execute the HTTP Get. When the Request finished, another Route is called to Post the return on Artemis ActiveMQ.
The project is deployed on Wildfly 13.
The problem is:
Sometimes the routes simply freeze. Don't start after 15 minutes.
When I try to stop/start the route, I got the follow log:
[0m[0m08:27:45,230 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) There are 1 inflight exchanges: InflightExchange: [exchangeId=ID-exchange-ID, fromRouteId=Route1, routeId=GetDataAutoBySinceTime, nodeId=toD7, elapsed=0, duration=216958569]
[0m[0m08:27:46,231 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) Waiting as there are still 1 inflight and pending exchanges to complete, timeout in 299 seconds. Inflights per route: [Route1 = 1]
[0m[0m08:27:46,231 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) There are 1 inflight exchanges: InflightExchange: [exchangeId=ID-exchange-ID, fromRouteId=Route1, routeId=GetDataAutoBySinceTime, nodeId=toD7, elapsed=0, duration=216959570]
[0m[0m08:27:47,231 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) Waiting as there are still 1 inflight and pending exchanges to complete, timeout in 298 seconds. Inflights per route: [Route1 = 1]
I don't know if some processes are stuck making it impossible another processes to start.
I thought to remove the generic routes (PostMessageInActiveMQ and
GetDataAutomaticallyBySinceTime and to implements the same code in another routes (Route1, Route2 and Route3) but I don't think this is the best approach.
Routes:
Route1 (Route2 and Route3 are almost the same, just change properties values)
from("timer:Route1Timer?period=15m")
.routeId("Route1")
.autoStartup(false)
.setProperty("targetAddress", simple("hostname.route1"))
.process(new GetCurrentDate())
.setProperty("query",
simple("DataQuery%26URI=Route1%26format=xml%26Mode=since-time%26p1=${header.currentDate}"))
.to("direct:GetDataAutoBySinceTime");
GetDataAutomaticallyBySinceTime
from("direct:GetDataAutoBySinceTime")
.routeId("GetDataAutoBySinceTime")
.autoStartup(true)
.removeHeaders("*")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.toD("http4:${header.targetAddress}/command=${header.query}%26httpClient.socketTimeout=3000")
.convertBodyTo(String.class, "utf-8")
.to("direct:PostMessageInActiveMQ");
PostMessageInActiveMQ
CamelArtemisComponent components = new CamelArtemisComponent();
getContext().addComponent("artemis", components.getArtemisComponent());
from("direct:PostMessageInActiveMQ")
.routeId("PostMessageInActiveMQ")
.autoStartup(true)
.convertBodyTo(String.class, "utf-8")
.inOnly("artemis:ARTEMIS.QUEUE");
Entire code: https://github.com/vitorvr/camel-example
EDIT:
Camel Version: 2.22.0

Apache camel how to implement an optional consumer for a wire tap

I set up some routes (Camel 2.22.1) that uses wire tap to log some stuff into a Mongo db.
from(DIRECT_NEXT).process(sendFile)
.wireTap( "direct:count-fetch?failIfNoConsumers=false" )
as you see i am using failIfNoConsumers=false.
from(COUNT_FETCH)
.routeId( MONGO_COUNT_FETCH_ROUTEID )
.autoStartup( false )
.process(countFetchProcessor)
.to(persistenceEndpoints.updateImage())
.log(LoggingLevel.DEBUG, "Counted fetch.");
The mongo DB is an optional component, the whole application will run without it.
I am using Mongo'S ServerMonitorListener to check if Mongo is available. I suspend or resume the rout using Camel's ControlBus accordingly.
All is running fine!
My Problem is that Camel tries to send the exchanges to the not running routes for 30s:
...
[DEBUG] 2019-01-03 14:02:45.848 [Camel (camel-1) thread #23 - WireTap] DirectBlockingProducer - Waited 20025 for consumer to be ready
...
Why the producer blocks? The default value for "block" should be false?!
And after it we see of course an exception:
No consumers available on endpoint: direct://count-fetch?failIfNoConsumers=false
What is the best approach to let camel discard the exchange immediately (how to set the time out?) and don't throw any exception (because it is normal application behavior, exception will only slow down)?
UPDATE:
here is the complete exception:
[ERROR] 2019-01-07 10:21:22.702 [Camel (camel-1) thread #4 - WireTap] DefaultErrorHandler - Failed delivery for (MessageId: ID-moritz-1546852848013-0-3 on ExchangeId: ID-moritz-1546852848013-0-2). Exhausted after delivery attempt: 1 caught: org.apache.camel.component.direct.DirectConsumerNotAvailableException: No consumers available on endpoint: direct://update-all?failIfNoConsumers=false. Exchange[ID-moritz-1546852848013-0-2]
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[route4 ] [route4 ] [timer://updateAll ] [ 30065]
[route4 ] [log1 ] [log ] [ 1]
[route4 ] [to3 ] [direct:updateAll ] [ 19]
[route5 ] [process2 ] [Processor#0x4e92466a ] [ 9]
[route5 ] [process3 ] [Processor#0x1b29d52b ] [ 7]
[route5 ] [wireTap1 ] [wireTap[direct:update-all?failIfNoConsumers=false] ] [ 1]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.apache.camel.component.direct.DirectConsumerNotAvailableException: No consumers available on endpoint: direct://update-all?failIfNoConsumers=false. Exchange[ID-moritz-1546852848013-0-2]
at org.apache.camel.component.direct.DirectBlockingProducer.getConsumer(DirectBlockingProducer.java:67) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.direct.DirectBlockingProducer.process(DirectBlockingProducer.java:53) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.SendDynamicProcessor$1.doInAsyncProducer(SendDynamicProcessor.java:178) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:445) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:160) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:160) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:155) [camel-core-2.22.1.jar:2.22.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Make sure to check the document for the version of Camel you use, which is 2.22.x
There you can see the block is default enabled: https://github.com/apache/camel/blob/camel-2.22.x/camel-core/src/main/docs/direct-component.adoc

Gatling active users drops to negative for a longer run after timeout exception

I'm running a simulation using gatling.
5 users per second for 150 mins.
After a certain exception:
15:58:19.643 [WARN ] i.g.h.e.GatlingHttpListener - Request 'facebook_outbound_msg' failed for user 29989
javax.net.ssl.SSLException: handshake timed out
at io.netty.handler.ssl.SslHandler.handshake(...)(Unknown Source)
15:58:19.643 [ERROR] i.g.h.e.r.DefaultStatsProcessor - Request 'facebook_outbound_msg' failed for user 29989: j.n.s.SSLException: handshake timed out
15:58:19.643 [ERROR] i.g.h.c.i.DefaultHttpClient - Failed to install SslHandler
15:58:19.678 [WARN ] i.g.h.e.GatlingHttpListener - Request 'facebook_inbound_msg' failed for user 29984
javax.net.ssl.SSLException: handshake timed out
at io.netty.handler.ssl.SslHandler.handshake(...)(Unknown Source)
15:58:19.678 [ERROR] i.g.h.e.r.DefaultStatsProcessor - Request 'facebook_inbound_msg' failed for user 29984: j.n.s.SSLException: handshake timed out
the number of active users dropped to -1, then everytime this exception happens, the number of users keeps dropping.
Example:
---- FacebookOutboundSimulation ------------------------------------------------
[############################################ ] 59%
waiting: 18061 / active: -3 / done: 26942
---- FacebookInboundMessageSimulation ------------------------------------------
[############################################ ] 59%
waiting: 18061 / active: -3 / done: 26942
================================================================================
Why does this happen and how to fix this?
The issue was with gatling-charts-highcharts-bundle-3.0.0-RC1 when I switched to gatling-charts-highcharts-bundle-3.0.0-RC3 that was released on 2nd October 2018 it got resolved.
Sidenote: Use AsJson instead of AsJSON.

An error occurred while executing user supplied JavaScript

I am getting this error while testing appium for ios :
> Exception in thread "main" org.openqa.selenium.WebDriverException: An
> error occurred while executing user supplied JavaScript. (WARNING: The
> server did not provide any stacktrace information) Command duration or
> timeout: 5.15 seconds Build info: version: '2.52.0', revision:
> '4c2593c', time: '2016-02-11 19:03:33' System info: host:
> 'users-iMac.local', ip: '192.168.1.16', os.name: 'Mac OS X', os.arch:
> 'x86_64', os.version: '10.10.5', java.version: '1.7.0_80' Session ID:
> cdcaf9fa-3530-4b09-8bca-c30c42d6c29e Driver info:
> io.appium.java_client.ios.IOSDriver Capabilities
> [{platformVersion=9.2, platform=MAC, databaseEnabled=false,
> javascriptEnabled=true, platformName=iOS, deviceName=iPad 2,
> browserName=iOS, webStorageEnabled=false,
> bundleId=com.InovaSquad.Artinove, networkConnectionEnabled=false,
> desired={platformVersion=9.2, deviceName=iPad 2, platformName=iOS,
> bundleId=com.InovaSquad.Artinove}, locationContextEnabled=false,
> warnings={}, takesScreenshot=true}] at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at
> org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206)
> at
> org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158)
> at
> org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:678)
> at
> io.appium.java_client.DefaultGenericMobileDriver.execute(DefaultGenericMobileDriver.java:42)
> at io.appium.java_client.AppiumDriver.execute(AppiumDriver.java:1) at
> io.appium.java_client.ios.IOSDriver.execute(IOSDriver.java:1) at
> org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:327)
> at
> io.appium.java_client.DefaultGenericMobileElement.execute(DefaultGenericMobileElement.java:44)
> at io.appium.java_client.MobileElement.execute(MobileElement.java:1)
> at io.appium.java_client.ios.IOSElement.execute(IOSElement.java:1) at
> org.openqa.selenium.remote.RemoteWebElement.sendKeys(RemoteWebElement.java:122)
> at test1.FirstTest.main(FirstTest.java:49)
FirstTest.java
package test1;
import java.net.MalformedURLException;
import java.net.URL;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import io.appium.java_client.AppiumDriver;
import io.appium.java_client.ios.IOSDriver;
import io.appium.java_client.ios.IOSElement;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
/*
#author:Hajar
11/03/2016
*/
public class FirstTest {
public static AppiumDriver dr;
public static void main(String[] args)
throws MalformedURLException, InterruptedException{
DesiredCapabilities capabilities = new DesiredCapabilities();
// Platform de l'emulateur
capabilities.setCapability("platformName","iOS");
// La version de l'emulateur
capabilities.setCapability("platformVersion", "9.2");
// le nom de l'emulateur
capabilities.setCapability("deviceName","iPad 2");
capabilities.setCapability("bundleId","com.InovaSquad.Artinove");
dr=new IOSDriver(new URL("http://127.0.0.1:4723/wd/hub"),capabilities);
dr.findElement(By.xpath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIALink[2]/UIAStaticText[1]")).click();
dr.findElement(By.xpath("//UIAApplication[1]/UIAWindow[1]/UIAScrollView[2]/UIAWebView[1]/UIATextField[1]")).sendKeys("ggg");
}
}
Server logs :
info : [debug] Sending command to instruments:
au.getElementByIndexPath('/0/0/1/0/1') info: [IOS_SYSLOG_ROW ] Mar 18
15:39:46 users-iMac mstreamd[70568]: (Note ) PS: Media stream daemon
stopping. info: [IOS_SYSLOG_ROW ] Mar 18 15:39:46 users-iMac
mstreamd[70568]: (Note ) AS: :
Shared Streams daemon has shut down. info: [IOS_SYSLOG_ROW ] Mar 18
15:39:46 users-iMac mstreamd[70568]: (Warn ) mstreamd: mstreamd
shutting down.
info: [debug] [INST] 2016-03-18 15:39:46 +0000 Debug: Got new command
7 from instruments: au.getElementByIndexPath('/0/0/1/0/1') info:
[debug] [INST] 2016-03-18 15:39:46 +0000 Debug: evaluating
au.getElementByIndexPath('/0/0/1/0/1')
info: [debug] Socket data received (38 bytes)
info: [debug] Socket data being routed. info: [debug] Got result from
instruments: {"status":0,"value":{"ELEMENT":"1"}} info: [debug]
Responding to client with success:
{"status":0,"value":{"ELEMENT":"1"},"sessionId":"d860df3d-b4be-46a5-8ee4-de67f7d04e55"}
info: <-- POST
/wd/hub/session/d860df3d-b4be-46a5-8ee4-de67f7d04e55/element 200
2387.424 ms - 87 {"status":0,"value":{"ELEMENT":"1"},"sessionId":"d860df3d-b4be-46a5-8ee4-de67f7d04e55"}
info: --> POST
/wd/hub/session/d860df3d-b4be-46a5-8ee4-de67f7d04e55/element/1/value
{"id":"1","value":["ggg"]} info: [debug] Pushing command to appium
work queue: "au.getElement('1').setValueByType('ggg')" info: [debug]
Sending command to instruments:
au.getElement('1').setValueByType('ggg') info: [debug] [INST]
2016-03-18 15:39:46 +0000 Debug: evaluation finished info: [debug]
[INST] 2016-03-18 15:39:46 +0000 Debug: responding with: info: [debug]
[INST] 2016-03-18 15:39:46 +0000 Debug: Running system command #8:
/Applications/Appium.app/Contents/Resources/node/bin/node
/Applications/Appium.app/Contents/Resources/node_modules/appium/submodules/appium-uiauto/bin/command-proxy-client.js
/tmp/instruments_sock 2,{"status":0,"value":{"ELEMENT":"1"}}...
info: [debug] [INST] 2016-03-18 15:39:47 +0000 Debug: Got new command
8 from instruments: au.getElement('1').setValueByType('ggg')
info: [debug] [INST] 2016-03-18 15:39:47 +0000 Debug: evaluating
au.getElement('1').setValueByType('ggg') info: [debug] [INST]
2016-03-18 15:39:47 +0000 Debug: No keyboard found. Tapping to make
visible. info: [debug] [INST] 2016-03-18 15:39:47 +0000 Debug:
target.frontMostApp().elements()[0].elements()[1].elements()[0].elements()[1].tap()
info: [IOS_SYSLOG_ROW ] Mar 18 15:39:47 users-iMac assertiond[70583]:
assertion failed: 14F27 13C75: assertiond + 13207
[792516D1-991E-36EE-8454-16F8220758D6]: 0x1 info: [debug] [INST]
2016-03-18 15:39:48 +0000 Debug: Still no keyboard found. Tapping
again to make visible.
info: [debug] [INST] 2016-03-18 15:39:48 +0000 Debug:
target.frontMostApp().elements()[0].elements()[1].elements()[0].elements()[1].tap()
info: [IOS_SYSLOG_ROW ] Mar 18 15:39:50 --- last message repeated 3
times --- info: [IOS_SYSLOG_ROW ] Mar 18 15:39:50 users-iMac
Artinove[70619]: mousedown stop event
info: [IOS_SYSLOG_ROW ] Mar 18 15:39:50 users-iMac assertiond[70583]:
assertion failed: 14F27 13C75: assertiond + 13207
[792516D1-991E-36EE-8454-16F8220758D6]: 0x1
info: [debug] [INST] 2016-03-18 15:39:50 +0000 Debug:
target.frontMostApp().keyboard().typeString("g")
info: [debug] [INST] 2016-03-18 15:39:51 +0000 Debug:
target.frontMostApp().keyboard() failed to tap 'g'
info: [debug] [INST] 2016-03-18 15:39:51 +0000 Debug: Error typing
'g': VerboseError: target.frontMostApp().keyboard() failed to tap 'g'
info: [IOS_SYSLOG_ROW ] Mar 18 15:39:51 --- last message repeated 1
time --- info: [IOS_SYSLOG_ROW ] Mar 18 15:39:51 users-iMac
kbd[70636]: table drop: 101
info: [debug] [INST] 2016-03-18 15:39:51 +0000 Debug: Retrying...
info: [debug] [INST] 2016-03-18 15:39:51 +0000 Debug:
target.frontMostApp().keyboard().typeString("g")
info: [debug] [INST] 2016-03-18 15:39:52 +0000 Debug:
target.frontMostApp().keyboard() failed to tap 'g'
info: [debug] Socket data received (76 bytes)
Any assistance would be greatly appreciated!

Unable to process large files in camel

I am trying to do simple transformation on a Csv file.But my programm is getting stuck and not giving any output and on console its printing something like below.
22:38:02.001 [main] INFO o.a.camel.impl.DefaultCamelContext - Apache Camel 2.15.2 (CamelContext: camel-1) is shutting down
22:38:02.135 [main] INFO o.a.c.impl.DefaultShutdownStrategy - Starting to graceful shutdown 1 routes (timeout 300 seconds)
22:38:02.167 [main] DEBUG o.a.c.i.DefaultExecutorServiceManager - Created new ThreadPool for source: org.apache.camel.impl.DefaultShutdownStrategy#65ead16a with name: ShutdownTask. -> org.apache.camel.util.concurrent.RejectableThreadPoolExecutor#52c0a65f[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0][ShutdownTask]
22:38:02.173 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 routes to shutdown
22:38:02.177 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - Route: route1 suspended and shutdown deferred, was consuming from: Endpoint[file:///home/cloudera/Desktop/camelinput/?delay=15m&noop=true]
22:38:02.177 [Camel (camel-1) thread #1 - ShutdownTask] INFO o.a.c.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 300 seconds.
22:38:02.179 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 inflight exchanges:
InflightExchange: [exchangeId=ID-quickstart-cloudera-40574-1441345060577-0-2, fromRouteId=route1, routeId=route1, nodeId=unmarshal1, elapsed=10787, duration=10791]
22:38:05.436 [Camel (camel-1) thread #1 - ShutdownTask] INFO o.a.c.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 299 seconds.
22:38:05.437 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 inflight exchanges:
InflightExchange: [exchangeId=ID-quickstart-cloudera-40574-1441345060577-0-2, fromRouteId=route1, routeId=route1, nodeId=unmarshal1, elapsed=14045, duration=14049]
22:38:08.201 [Camel (camel-1) thread #1 - ShutdownTask] INFO o.a.c.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 298 seconds.
22:38:08.202 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 inflight exchanges:
InflightExchange: [exchangeId=ID-quickstart-cloudera-40574-1441345060577-0-2, fromRouteId=route1, routeId=route1, nodeId=unmarshal1, elapsed=16810, duration=16814]
Actually the same program worked for small file but when I try to do with large file I am getting this issue.I think it may problem with Threads .Please Help me out to figure out the issue.
Following is my Program
Main Class
TestRouter myRoute = new TestRouter();
HDFSTransfer hdfsTransfer = new HDFSTransfer();
String copy = hdfsTransfer.copyToLocal(
"hdfs://localhost:8020",
"/user/cloudera/input/CamelTestIn.csv",
"/home/cloudera/Desktop/camelinput/");
boolean flag = false;
if ("SUCCESS".equals(copy)) {
myContext.addRoutes(myRoute);
// Launching the context
myContext.start();
// Pausing to let the route do its work
Thread.sleep(10000);
myContext.stop();
flag = true;
}
if (flag) {
hdfsTransfer.moveFile(
"hdfs://localhost:8020",
"file:/home/cloudera/Desktop/camelout/out.csv",
"/user/cloudera/output/");
}
RouterBuilder Class
{
CsvDataFormat csv = new CsvDataFormat();
from("file:/home/cloudera/Desktop/camelinput/?noop=true&delay=15m")
.unmarshal(csv)
.convertBodyTo(List.class)
.process(new Processor() {
#Override
public void process(Exchange msg) throws Exception {
List<List<String>> data = (List<List<String>>) msg.getIn().getBody();
for (List<String> line : data) {
// Checks if column two contains text STANDARD
// and alters its value to DELUXE.
// System.out.println("line "+line);
/*
if("Aug-04".equalsIgnoreCase(line.get(6))){
line.set(6, "04-August");}
*/
}
}
}).marshal(csv)
.to("file:/home/cloudera/Desktop/camelout/?fileName=out.csv")
.log("done.").end();
}
If you have a bigger file then you need to sleep for longer than 10 seconds to let it have time to process the file.
Also mind that your current route reads the file into memory when means you can run out of memory if the file is very big.
See the lazyLoad option on: http://camel.apache.org/csv.html
Also if all your route is doing is to change some text in a big file, then there is better and faster ways doing that than maybe a Camel route.

Resources