I'm using the camel-sftp component to upload a file to an SFTP server.
The code is simple:
File source = new File(path);
final String sftpUri = "sftp://" + userId + "#" + serverAddress + "/" + remoteDirectory+"?password="+pwd;
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
from("file:/" + path).to(sftpUri);
}
});
context.start();
Thread.sleep(10000);
context.stop();
However, camel has problems finding the sftp component. Activating the debug logs in Camel it complains:
| 62 - org.apache.camel.camel-core - 2.13.2 | Using ComponentResolver: org.apache.camel.impl.DefaultComponentResolver#29668a43 to resolve component with name: sftp
| 62 - org.apache.camel.camel-core - 2.13.2 | Found component: sftp in registry: null
| 62 - org.apache.camel.camel-core - 2.13.2 | Apache Camel 2.13.2 (CamelContext: camel-16) is shutting down
Any ideas why Camel is behaving this way? In fact, running this code in a standalone application (a Java class with a main method) works correctly.
And I can see:
11:22:13.237 [main] DEBUG o.a.c.impl.DefaultComponentResolver - Found component: sftp in registry: null
11:22:13.239 [main] DEBUG o.a.c.impl.DefaultComponentResolver - Found component: sftp via type: org.apache.camel.component.file.remote.SftpComponent via: META-INF/services/org/apache/camel/component/sftp
Inside of Karaf, though, only the first line appears, for some reason or other it does not find META-INF/services/org/apache/camel/component/sftp and as a result the sftp component is not found.
If you run Camel inside OSGi, you should use the OSGi CamelContext from camel-core-osgi. And then there is a few more steps to setup this for OSGi.
Though its often easier to use a OSGi blueprint application and bootstrap Camel in the blueprint xml file, which does this correctly.
But for Java code its some manual process. In the upcoming Camel 2.15 release there is a new camel-scr component for working with OSGi and SCR (Declarative Services) which makes it easier to do java code with Camel in OSGi using camel-scr.
I would suggest to check its current source code for inspiration how you can setup Camel in OSGi from Java code
https://github.com/apache/camel/tree/master/components/camel-scr
Related
My Spring boot camel application uses camel-corda component and failing to start when corda node (RPC connection) is not up and running
My current camel route is
#Component
class CordaOpsRouteBuilder() : RouteBuilder() {
override fun configure() {
from("timer://terminate?repeatCount=1&delay=20").autoStartup("{{corda.terminate.node}}")
.to("direct:terminate-node")
from("direct:terminate-node")
.log("Draining and shutting down node")
.to("corda://{{corda.rpc.username}}:{{corda.rpc.password}}#{{corda.rpc.host}}:{{corda.rpc.port}}?operation=TERMINATE")
.delay(10000).asyncDelayed()
.to("direct:shutdown")
from ("direct:shutdown")
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.to("http://localhost:{{server.port}}/actuator/shutdown")
}
}
Stack trace :
Caused by: org.apache.camel.RuntimeCamelException: net.corda.client.rpc.RPCException: Cannot connect to server(s). Tried with all available servers.
at org.apache.camel.RuntimeCamelException.wrapRuntimeCamelException(RuntimeCamelException.java:52) ~[camel-api-3.0.0-RC3.jar:3.0.0-RC3]
at org.apache.camel.support.ChildServiceSupport.start(ChildServiceSupport.java:63) ~[camel-support-3.0.0-RC3.jar:3.0.0-RC3]
How can handle above RuntimeCamelException during the startup?
Please use lazyStartProducer=true parameter on the corda endpoint to make sure the route starts even if corda is not available (a lazy start).
https://camel.apache.org/components/latest/corda-component.html
I observe, that errors in the route definition lead to a silent shutdown of the application, caused by Camel. How can I configure Camel to telling me what exactly it dislikes.
Simple example: Assigning duplicate route names
If I define two routes with different names [by using: .id ("route name")], then the application starts and reports readiness.
If I mistakenly use a name twice, the application fails to start completely and Camel announces:
Apache Camel 2.22.0 (CamelContext: camel-1) is shutting down
Apache Camel 2.22.0 (CamelContext: camel-1) uptime 0.332 seconds
Apache Camel 2.22.0 (CamelContext: camel-1) is shutdown in 0.017 seconds
Stopping service [Tomcat]
HikariPool-1 - Shutdown initiated ...
HikariPool-1 - Shutdown completed.
Process finished with exit code 0
The exit code 0 seems to suggest a non-exceptional shutdown.
The same early shutdown happened, when I added a CronScheduledRoutePolicy to a route. [e.g. with .routePolicy (policy)]
It shuts down if I add a default instance or if I make any settings to the policy.
I increased the log level and got significantly more background noise, but no new findings. [using application.yml: logging: level: ROOT: DEBUG]
A try catch around the route definition didn't help. No exception was caught at the time of the route definition.
I tried to set the CronScheduledRoutePolicy an exception handler. [E.g. with policy.setExceptionHandler (myCamelExceptionHandlerLoggingEverything)]
Nothing got logged. No breakpoint was hit.
I would be very grateful for your help or references to any solutions.
Ralf
There are multiple ways to print an exact error and it depends on how developer design app/class with the camel route. I have mentioned below example which will give exact error you are looking for.
public class CamelPrintingExample {
public static void main(String[] args) throws Exception {
CamelContext camelContext = new DefaultCamelContext();
try {
camelContext.addRoutes(new RouteBuilder() {
public void configure() {
from("file:C:\\input\\").routeId("demo")
.to("file:C:\\output\\").end();
from("file:C:\\input\\").routeId("demo")
.to("file:C:\\output\\").end();
}
});
camelContext.start();
Thread.sleep(300000);
camelContext.stop();
} catch (Exception camelException) {
camelException.printStackTrace();
}
}
}
By running this I got below error.
org.apache.camel.FailedToStartRouteException: Failed to start route demo because of duplicate id detected: demo. Please correct ids to be unique among all your routes.
You can try defining a custom shutdown strategy. Read here
Try to add this in your route and then start it:
getContext().setTracing(true);
P.S. a problem could also be in your error handler
I'm new to apache camel and apache kafka and doing a small POC for my project. I am getting following issue error log when trying to read from kafka using Camel-kafka component.
[2016-01-20 08:47:10,979] INFO Closing socket connection to /127.0.0.1. (kafka.network.Processor)
[2016-01-20 08:47:44,643] INFO Closing socket connection to /127.0.0.1. (kafka.network.Processor)
[2016-01-20 08:47:54,545] ERROR Closing socket for /127.0.0.1 because of error (kafka.network.Processor)
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:65)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:470)
at kafka.api.TopicDataSend.writeTo(FetchResponse.scala:123)
at kafka.network.MultiSend.writeTo(Transmission.scala:101)
at kafka.api.FetchResponseSend.writeTo(FetchResponse.scala:231)
at kafka.network.Processor.write(SocketServer.scala:472)
at kafka.network.Processor.run(SocketServer.scala:342)
at java.lang.Thread.run(Thread.java:745)
My java code is as follows:
public class Main {
public static void main(String[] args) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("kafka:127.0.0.1:9092?topic=TEST&zookeeperHost=localhost&zookeeperPort=2181&groupId=group1")
/*.marshal(xmlJsonFormat)*/
.process(new XmlToJson())
/*.to("kafka:localhost:9092?topic=TestJson&zookeeperHost=localhost&zookeeperPort=2181&groupId=group1");*/
.to("file:/Users/himanshu/Desktop/TransCamelFuse/test.txt");
}
});
context.start();
Thread.sleep(10000);
context.stop();
}
}
I've put some txt from kafka producer console tool and trying to read using camel component of kafka.
It's not error.Giving this error because of your give data type. You are giving data to Kafka manually one by one.
You need to make sure the kafka server and kafka client versions are compatible to each i faced the same my kafka server was 0.8 and some spring beans uses kafka 2.4 libraries so whenever 2.4 client request for any kafka 0.8 operation server.log record this error
Solution
Be sure in code none of beans uses incompatible client for communication.
I'm using apache camel 2.12.1 to create a route and then move some files in my local directory, the exmple runs fine but the files are never moved, this is the code for the class.
public class MoveFilesTest {
private static final Log LOG = LogFactory.getLog(MoveFilesTest.class);
public static void main(String args[]) throws Exception {
LOG.debug("create CamelContext");
CamelContext context = new DefaultCamelContext();
// add our route to the CamelContext
context.addRoutes(new RouteBuilder() {
File file = null;
public void configure() {
from("file:data/inbox?delay=100&noop=true")
.process( new Processor() {
public void process(Exchange msg) throws Exception {
File file = msg.getIn().getBody(File.class);
LOG.debug("Processing file: " + file.getName());
}
})
.to("file:data/outbox").end();
}
});
LOG.debug("start the route and let it do its work");
context.start();
context.stop();
}
}
as a note, this code just to work, now i'm working on mac os x 10.7, this is the debug log. i added the noop=false and the delete=true, but the result is the same. thank you
DEBUG [main] (MoveFilesTest.java:24) - create CamelContext
DEBUG [main] (MoveFilesTest.java:45) - start the route and let it do its work
INFO [main] (DefaultCamelContext.java:1498) - Apache Camel 2.12.1 (CamelContext: camel-1) is starting
INFO [main] (ManagedManagementStrategy.java:187) - JMX is enabled
INFO [main] (DefaultTypeConverter.java:50) - Loaded 176 type converters
INFO [main] (DefaultCamelContext.java:1689) - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
INFO [main] (FileEndpoint.java:83) - Endpoint is configured with noop=true so forcing endpoint to be idempotent as well
INFO [main] (FileEndpoint.java:89) - Using default memory based idempotent repository with cache max size: 1000
INFO [main] (DefaultCamelContext.java:2183) - Route: route1 started and consuming from: Endpoint[file://data/inbox?delay=100&noop=true]
INFO [main] (DefaultCamelContext.java:1533) - Total 1 routes, of which 1 is started.
INFO [main] (DefaultCamelContext.java:1534) - Apache Camel 2.12.1 (CamelContext: camel-1) started in 8.936 seconds
INFO [main] (DefaultCamelContext.java:1706) - Apache Camel 2.12.1 (CamelContext: camel-1) is shutting down
INFO [main] (DefaultShutdownStrategy.java:172) - Starting to graceful shutdown 1 routes (timeout 300 seconds)
INFO [Camel (camel-1) thread #2 - ShutdownTask] (DefaultShutdownStrategy.java:600) - Route: route1 shutdown complete, was consuming from: Endpoint[file://data/inbox?delay=100&noop=true]
INFO [main] (DefaultShutdownStrategy.java:217) - Graceful shutdown of 1 routes completed in 0 seconds
INFO [main] (DefaultCamelContext.java:1780) - Apache Camel 2.12.1 (CamelContext: camel-1) uptime 8.953 seconds
INFO [main] (DefaultCamelContext.java:1781) - Apache Camel 2.12.1 (CamelContext: camel-1) is shutdown in 0.013 seconds
Yes, you start Camel and stop it immediately. So, when you put a file to a folder. It wont process cuz camel is already stopped.
Camel contains Main implementation to keep Camel running in standalone application.
There is link: http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html
CamelContext.start does not block, so basically you are starting the context and then immediately stopping it. You need to wait or block on something until the context should stop. You can reference this thread for some ways of doing this.
I had a similar problem but that was to do with streamCaching() not getting set properly and so the code below was failing
context.getStreamCachingStrategy().setSpoolDirectory(spoolDirectory);
from(uri.toString()).streamCaching().to(destination);
I set the Streaming directly on CamelContext and that solved the problem
CamelContext context = getContext();
context.setStreamCaching(true);
context.getStreamCachingStrategy().setSpoolDirectory(localSpoolDirectory);
context.getStreamCachingStrategy().setSpoolThreshold(Long.parseLong(spoolThreshold.trim()));
context.getStreamCachingStrategy().setBufferSize(Integer.parseInt(bufferSize.trim()));
We are using Apache CXF 2.5.2 for webservice client proxies. We use weblogic 10.3.4. To override the CXF logger we use the following option:
-Dorg.apache.cxf.Logger=org.apache.cxf.common.logging.Slf4jLogger
For the org.apache.cxf.common.logging.Slf4jLogger, we've included the cxf-common-utilities-2.5.2 in our build.
When we try to deploy to weblogic we get the following exception:
[ERROR] Target state: deploy failed on Server AdminServer
[ERROR] java.lang.NoClassDefFoundError: weblogic/wsee/jaxws/spi/WLSProvider
According to documentation, your WebService client should have wseeclient.jar as it's runtime dependency. If the problems is still there, then include wlfullclient.jar (see Creating a wlfullclient.jar for JDK 1.6 client applications ).