How to configure REST properties in Java-only Camel Main? - apache-camel

I'm working on a simple "hello-world" Camel3 (Camel version 3.4.3) REST API example with the following setup:
REST API is described in OpenAPI specification (Swagger v2.0)
The API spec is converted to Java DSL with camel-restdsl-swagger-plugin
Using Camel Main
100% Java (i.e. no XML)
The example below works as expected but I fail to set jetty port. Therefore on every run the web server runs on a random port. How do I set the port (e.g. to 8081) in the Java code ? It also seems to be possible to set the port in application.properties file. Example of that option is also appreciated.
// https://camel.apache.org/components/latest/others/main.html
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.main.Main;
import org.apache.camel.main.RestConfigurationProperties;
import rest.dsl.generated.Api101;
public class App {
public static void main(String[] args) throws Exception {
Main main = new Main();
// How do I set the rest properties ?
// The code below doesn't seem to have effect.
RestConfigurationProperties p = main.configure().rest()
.withComponent("jetty")
.withHost("localhost")
.withPort(8081)
;
main.configure().addConfiguration(p);
// The route generated by camel-restdsl-swagger-plugin
main.configure().addRoutesBuilder(
new Api101()
);
// The actual route.
main.configure().addRoutesBuilder(
new RouteBuilder() {
public void configure() {
from("direct:rest1")
.routeId("Rest1Route")
.log("START:")
.setBody(constant("{hello: \"Hello, World!\"}"))
.log("END:")
;
}
}
);
main.run(args);
}
}
Generated Api101 code:
package rest.dsl.generated;
import javax.annotation.Generated;
import org.apache.camel.builder.RouteBuilder;
/**
* Generated from Swagger specification by Camel REST DSL generator.
*/
#Generated("org.apache.camel.generator.swagger.PathGenerator")
public final class Api101 extends RouteBuilder {
/**
* Defines Apache Camel routes using REST DSL fluent API.
*/
public void configure() {
rest("/api101")
.get("/hello")
.description("Basic Hello World")
.to("direct:rest1");
}
}
Compilation & run example:
/tmp/openapi$ mvn clean compile exec:java
[INFO] Scanning for projects...
[INFO]
[INFO] ----------------------< com.stackoverflow:openapi >----------------------
[INFO] Building openapi 1.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) # openapi ---
[INFO] Deleting /tmp/openapi/target
[INFO]
[INFO] --- camel-restdsl-swagger-plugin:3.4.3:generate-with-dto (default) # openapi ---
[INFO] reading from /tmp/openapi/swagger.yaml
[INFO] Generating Camel DSL source in directory: /tmp/openapi/target/generated-sources/restdsl-swagger
[INFO] Generating DTO classes using io.swagger:swagger-codegen-maven-plugin:2.4.12
[INFO] reading from /tmp/openapi/swagger.yaml
[WARNING] Output directory does not exist, or is inaccessible. No file (.swagger-codegen-ignore) will be evaluated.
[WARNING] 'host' not defined in the spec. Default to 'localhost'.
[INFO] writing file /tmp/openapi/target/generated-sources/swagger/src/main/java/io/swagger/client/model/InlineResponse200.java
[WARNING] 'host' not defined in the spec. Default to 'localhost'.
[INFO]
[INFO] --- build-helper-maven-plugin:3.2.0:add-source (default) # openapi ---
[INFO] Source directory: /tmp/openapi/target/generated-sources/restdsl-swagger added.
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # openapi ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) # openapi ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 3 source files to /tmp/openapi/target/classes
[INFO]
[INFO] --- exec-maven-plugin:3.0.0:java (default-cli) # openapi ---
[ App.main()] BaseMainSupport INFO Using properties from: classpath:application.properties;optional=true
[ App.main()] DefaultRoutesCollector INFO No additional Camel XML routes discovered from: classpath:camel/*.xml
[ App.main()] DefaultRoutesCollector INFO No additional Camel XML rests discovered from: classpath:camel-rest/*.xml
[ App.main()] AbstractCamelContext INFO Apache Camel 3.4.3 (camel-1) is starting
[ App.main()] AbstractCamelContext INFO StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[ App.main()] JettyHttpComponent WARN JMX disabled in CamelContext. Jetty JMX extensions will remain disabled.
[ App.main()] log INFO Logging initialized #8774ms to org.eclipse.jetty.util.log.Slf4jLog
[ App.main()] Server INFO jetty-9.4.29.v20200521; built: 2020-05-21T17:20:40.598Z; git: 77c232aed8a45c818fd27232278d9f95a021095e; jvm 11.0.8+10-post-Ubuntu-0ubuntu120.04
[ App.main()] ContextHandler INFO Started o.e.j.s.ServletContextHandler#4d9f9a55{/,null,AVAILABLE}
[ App.main()] AbstractConnector INFO Started ServerConnector#157aa401{HTTP/1.1, (http/1.1)}{0.0.0.0:44267}
[ App.main()] Server INFO Started #8938ms
[ App.main()] InternalRouteStartupManager INFO Route: route1 started and consuming from: jetty:http://0.0.0.0:0/api101/hello
[ App.main()] InternalRouteStartupManager INFO Route: Rest1Route started and consuming from: direct://rest1
[ App.main()] AbstractCamelContext INFO Total 2 routes, of which 2 are started
[ App.main()] AbstractCamelContext INFO Apache Camel 3.4.3 (camel-1) started in 0.225 seconds
[ qtp1580225589-17] Rest1Route INFO START:
[ qtp1580225589-17] Rest1Route INFO END:
^C[ad #0 - CamelHangupInterceptor] DefaultMainShutdownStrategy INFO Received hang up - stopping the main instance.
[ App.main()] AbstractCamelContext INFO Apache Camel 3.4.3 (camel-1) is shutting down
[ad #0 - CamelHangupInterceptor] DefaultMainShutdownStrategy INFO Waiting for CamelContext to graceful shutdown, elapsed: 0.000 seconds
[ App.main()] DefaultShutdownStrategy INFO Starting to graceful shutdown 2 routes (timeout 45 seconds)
[el-1) thread #1 - ShutdownTask] DefaultShutdownStrategy INFO Route: Rest1Route shutdown complete, was consuming from: direct://rest1
[el-1) thread #1 - ShutdownTask] AbstractConnector INFO Stopped ServerConnector#157aa401{HTTP/1.1, (http/1.1)}{0.0.0.0:0}
[el-1) thread #1 - ShutdownTask] ContextHandler INFO Stopped o.e.j.s.ServletContextHandler#4d9f9a55{/,null,UNAVAILABLE}
[el-1) thread #1 - ShutdownTask] DefaultShutdownStrategy INFO Route: route1 shutdown complete, was consuming from: rest://get:/api101:/hello
[ App.main()] DefaultShutdownStrategy INFO Graceful shutdown of 2 routes completed in 0 seconds
[ App.main()] MainLifecycleStrategy INFO CamelContext: camel-1 has been shutdown, triggering shutdown of the JVM.
[ App.main()] AbstractCamelContext INFO Apache Camel 3.4.3 (camel-1) uptime 44.386 seconds
[ App.main()] AbstractCamelContext INFO Apache Camel 3.4.3 (camel-1) is shutdown in 0.037 seconds
/tmp/openapi$
As you can see from the above the web server is available in port 44267:
Started ServerConnector#157aa401{HTTP/1.1, (http/1.1)}{0.0.0.0:44267}
And can be successfully accessed:
$ sudo netstat -tulpn | grep /java
tcp6 0 0 :::44267 :::* LISTEN 16926/java
$ curl http://0.0.0.0:44267/api101/hello
{hello: "Hello, World!"}
$

I found an answer to my secondary question: this is the way to define the port with the properties:
$ cat src/main/resources/application.properties
camel.rest.port=8081
$
This is also the method how to set all the other configuration options mention in the documentation. Might be obvious for Java veterans, but it was not obvious for me.

Related

Apache Camel Route is working under Windows but not Linux

I am executing Camel code in Windows using Eclipse and it is working fine.
However, when I execute the same code in standalone from Linux, the route has print first log but when fetching file it stops without any error.
Here is my code:
from("timer://alertstrigtimer?period=90s&repeatCount=1")
.log(LoggingLevel.INFO, "*******************************Job-Alert-System: Started: alertstrigtimer******************************" + getFileURI(getWorkFilePath(), getWorkFileName()))
.pollEnrich(getFileURI(getWorkFilePath(), getWorkFileName()))
.log(LoggingLevel.INFO, "*******************************Job-Alert-System: Started: alertstrigtimer******************************" + getFileURI(getWorkFilePath(), getWorkFileName()))
.choice()
.when(header("CamelFileName").isNull())
.log(LoggingLevel.INFO, "No File")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
log.info("Job-Alert-System: No Date File Exist!!!! Calculate 15 Minutes Back and fetching data from Masterdata");
// Do something
}
})
.otherwise()
.log(LoggingLevel.INFO, "Job Alert System: Date File Loaded: ${header.CamelFileName} at ${header.CamelFileLastModified}")
.process(new Processor() {
// Do something by a processor
})
public static String getFileURI(String filePath, String fileName) {
return "file://" + filePath + "?fileName=" + fileName
+ "&preMove=$simple{file:onlyname.noext}.$simple{date:now:yyyy-MM-dd'T'hh-mm-ss}";
}
Here are my logs from the Linux environment:
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) is starting
[main] INFO org.apache.camel.management.ManagedManagementStrategy - JMX is enabled
[main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Type converters loaded (core: 194, classpath: 0)
[main] INFO org.apache.camel.impl.DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: timer://alertstrigtimer?period=90s&repeatCount=1
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: loadDataAndAlerts started and consuming from: direct://loadDataAndAlerts
[main] INFO org.apache.camel.impl.DefaultCamelContext - Total 2 routes, of which 2 are started
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) started in 0.664 seconds
[Camel (camel-1) thread #1 - timer://alertstrigtimer] INFO route1 - *******************************Job-Alert-System: Started: alertstrigtimer******************************file:///shared/wildfly/work-files/alerts?fileName=LastExecutionTime_JobAlerts.txt&preMove=.2020-10-12T06-48-16
It stops here. It creates a directory structure, but does not move forward.
Logs from My Local Machine:
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) is starting
[main] INFO org.apache.camel.management.ManagedManagementStrategy - JMX is enabled
[main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Type converters loaded (core: 194, classpath: 5)
[main] INFO org.apache.camel.impl.DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: timer://alertstrigtimer?period=90s&repeatCount=1
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: loadDataAndAlerts started and consuming from: direct://loadDataAndAlerts
[main] INFO org.apache.camel.impl.DefaultCamelContext - Total 2 routes, of which 2 are started
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.21.1 (CamelContext: camel-1) started in 0.845 seconds
[Camel (camel-1) thread #1 - timer://alertstrigtimer] INFO route1 - *******************************Job-Alert-System: Started: alertstrigtimer******************************file://null?fileName=null&preMove=null.2020-10-12T10-28-51
[Camel (camel-1) thread #1 - timer://alertstrigtimer] INFO route1 - Job Alert System: Date File Loaded: null.2020-10-12T10-28-51 at 0
It creates directory structure in addition to a file, but the file is not present and it moves forward.

Unable to consume messages with Camel 3.3 and Google Pubsub

I have a small groovy script with Apache camel that is attempting to pull messages from Google Pubsub. This works well on Camel 2.25.1, but camel 3.3.0 does not. Here is the script. I also looked at the Camel 2.x to 3.0 migration guide, but I am unable to find anything relevant to this issue, but perhaps I am mistaken and I would need some help.
My groovy version is 3.0.4 and I am using JDK 11.0.6
#Grapes([
#Grab(group='org.apache.camel', module='camel-core', version='3.3.0'),
#Grab(group='org.apache.camel', module='camel-google-pubsub', version='3.3.0'),
#Grab(group='org.slf4j', module='slf4j-api', version='1.7.30'),
#Grab(group='ch.qos.logback', module='logback-classic', version='1.2.3'),
])
import org.apache.camel.CamelContext
import org.apache.camel.impl.DefaultCamelContext
import org.apache.camel.builder.RouteBuilder
main()
def main() {
println "downloaded camel dependencies"
CamelContext context = new DefaultCamelContext()
context.addRoutes(new ReadFromQueueWithPubsub())
context.start()
addShutdownHook { context.stop() }
synchronized(this){ this.wait() }
}
class ReadFromQueueWithPubsub extends RouteBuilder {
#Override
void configure() {
from("google-pubsub://my_gcp_project:transaction-test-subscriber")
.to("log:input?showAll=true")
}
}
Here is the snippet of logs in debug mode. I can't see anything that's relevant here, but I may be mistaken.
2020-05-28 07:14:51 INFO [o.a.c.i.e.AbstractCamelContext ] - Apache Camel 3.3.0 (CamelContext: camel-1) is starting
2020-05-28 07:14:51 DEBUG [o.a.c.i.e.AbstractCamelContext ] - Using ClassResolver=org.apache.camel.impl.engine.DefaultClassResolver#25478603, PackageScanClassResolver=org.apache.camel.impl.engine.DefaultPackageScanClassResolver#3c7dbf1f, ApplicationContextClassLoader=null, RouteController=org.apache.camel.impl.engine.DefaultRouteController#2efbe938
2020-05-28 07:14:51 INFO [o.a.c.i.e.AbstractCamelContext ] - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
2020-05-28 07:14:51 DEBUG [o.a.c.i.e.AbstractCamelContext ] - Using HeadersMapFactory: org.apache.camel.impl.engine.DefaultHeadersMapFactory#12459991
2020-05-28 07:14:51 DEBUG [o.a.c.i.e.AbstractCamelContext ] - Using ReactiveExecutor: org.apache.camel.impl.engine.DefaultReactiveExecutor#43114981
2020-05-28 07:14:51 DEBUG [o.a.c.i.e.AbstractCamelContext ] - Warming up route id: route1 having autoStartup=true
2020-05-28 07:14:51 DEBUG [o.a.c.s.DefaultProducer ] - Starting producer: Producer[log://input?showAll=true]
2020-05-28 07:14:52 DEBUG [o.a.c.i.e.AbstractCamelContext ] - Route: route1 >>> Route[google-pubsub://my_gcp_project:transaction-test-subscriber -> null]
2020-05-28 07:14:52 DEBUG [o.a.c.i.e.AbstractCamelContext ] - Starting consumer (order: 1000) on route: route1
2020-05-28 07:14:52 DEBUG [o.a.c.s.DefaultConsumer ] - Init consumer: Consumer[google-pubsub://my_gcp_project:transaction-test-subscriber]
2020-05-28 07:14:52 DEBUG [o.a.c.s.DefaultConsumer ] - Starting consumer: Consumer[google-pubsub://my_gcp_project:transaction-test-subscriber]
2020-05-28 07:14:52 INFO [o.a.c.c.g.p.GooglePubsubConsumer ] - Starting Google PubSub consumer for my_gcp_project/transaction-test-subscriber
2020-05-28 07:14:52 DEBUG [o.a.c.i.e.BaseExecutorServiceManager ] - Created new ThreadPool for source: google-pubsub://my_gcp_project:transaction-test-subscriber with name: GooglePubsubConsumer[transaction-test-subscriber]. -> org.apache.camel.util.concurrent.RejectableThreadPoolExecutor#7d726fa1[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0][GooglePubsubConsumer[transaction-test-subscriber]]
2020-05-28 07:14:52 INFO [o.a.c.i.e.AbstractCamelContext ] - Route: route1 started and consuming from: google-pubsub://my_gcp_project:transaction-test-subscriber
2020-05-28 07:14:52 INFO [o.a.c.i.e.AbstractCamelContext ] - Total 1 routes, of which 1 are started
2020-05-28 07:14:52 INFO [o.a.c.i.e.AbstractCamelContext ] - Apache Camel 3.3.0 (CamelContext: camel-1) started in 0.078 seconds
2020-05-28 07:14:52 DEBUG [o.a.c.c.g.p.GooglePubsubConsumer ] - Subscribing to projects/my_gcp_project/subscriptions/transaction-test-subscriber
Closing this out: It looks like there's a bug open in JIRA for this CAMEL-15064 that affects versions >= 3.2

No plugin found for prefix 'gcloud' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo]

I'm trying to deploy and run Google-pubsub sample code in Google app engine by this read me file. While running the deploy command I encounter below build error.
E:\java\cloud-pubsub-samples-java-master>mvn gcloud:deploy -Dgcloud.version=1 -DskipTests=true
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] pubsub-pull-sample
[INFO] Example for the Google Cloud Pub/Sub on Google App Engine.
[INFO] Cloud Dataflow Examples
[INFO] cloud-pubsub-samples-java
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/m2e/lifecycle-mapping/1.0.0/lifecycle-mapping-1.0.0.pom
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no dependency information available
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/m2e/lifecycle-mapping/1.0.0/lifecycle-mapping-1.0.0.jar
[WARNING] Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be resolved: Could not find artifact org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in central (https://repo.maven.apache.org/maven2)
Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml
Downloading: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml
Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (13 KB at 9.8 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (20 KB at 8.9 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] pubsub-pull-sample ................................. SKIPPED
[INFO] Example for the Google Cloud Pub/Sub on Google App Engine. SKIPPED
[INFO] Cloud Dataflow Examples ............................ SKIPPED
[INFO] cloud-pubsub-samples-java .......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.167 s
[INFO] Finished at: 2016-08-21T22:20:45+05:30
[INFO] Final Memory: 13M/245M
[INFO] ------------------------------------------------------------------------
[ERROR] No plugin found for prefix 'gcloud' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (C:\Users\JAISON.G\.m2\repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoPluginFoundForPrefixException
This means that you haven't included the gcloud maven plugin in pom.xml. But you should no longer use gcloud, rather using appengine. See the documentation here.

Apache - Zeppelin build failed on Ubuntu

I am so stuck in this. I have followed each and every single instruction here (https://github.com/apache/incubator-zeppelin)
to install Apache-Zeppelin. I need this for Spark.
Below are various required applications installed in UBUNTU (VM)
hduser#HDPMaster:~$ java -version
java version "1.7.0_91"
OpenJDK Runtime Environment (IcedTea 2.6.3) (7u91-2.6.3-0ubuntu0.14.04.1)
OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode)
hduser#HDPMaster:~$ mvn --version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
Maven home: /usr/local/apache-maven-339
Java version: 1.7.0_91, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.19.0-43-generic", arch: "amd64", family: "unix"
hduser#HDPMaster:~$ node --version
v0.10.25
Cloned Zeppelin from Git Hub
git clone https://github.com/apache/incubator-zeppelin.git
Initially, I had Maven 3.0 version. One of the threads suggested to have a version > 3.3..Installed the latest version (3.3.9).
executed the below command which is still failing
mvn clean package -DskipTests -Pspark-1.5.2 -Phadoop-2.7.1
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 32.205 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 36.457 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 21.499 s]
[INFO] Zeppelin: Spark dependencies ....................... SUCCESS [ 51.142 s]
[INFO] Zeppelin: Spark .................................... SUCCESS [01:01 min]
[INFO] Zeppelin: Markdown interpreter ..................... SUCCESS [ 3.089 s]
[INFO] Zeppelin: Angular interpreter ...................... SUCCESS [ 2.736 s]
[INFO] Zeppelin: Shell interpreter ........................ SUCCESS [ 2.813 s]
[INFO] Zeppelin: Hive interpreter ......................... SUCCESS [ 12.357 s]
[INFO] Zeppelin: Apache Phoenix Interpreter ............... SUCCESS [ 15.315 s]
[INFO] Zeppelin: PostgreSQL interpreter ................... SUCCESS [ 4.681 s]
[INFO] Zeppelin: Tajo interpreter ......................... SUCCESS [ 6.417 s]
[INFO] Zeppelin: Flink .................................... FAILURE [ 6.133 s]
[INFO] Zeppelin: Apache Ignite interpreter ................ SKIPPED
[INFO] Zeppelin: Kylin interpreter ........................ SKIPPED
[INFO] Zeppelin: Lens interpreter ......................... SKIPPED
[INFO] Zeppelin: Cassandra ................................ SKIPPED
[INFO] Zeppelin: Elasticsearch interpreter ................ SKIPPED
[INFO] Zeppelin: web Application .......................... SKIPPED
[INFO] Zeppelin: Server ................................... SKIPPED
[INFO] Zeppelin: Packaging distribution ................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:17 min
[INFO] Finished at: 2016-01-13T06:12:12-08:00
[INFO] Final Memory: 122M/341M
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "spark-1.5.2" could not be activated because it does not exist.
[WARNING] The requested profile "hadoop-2.7.1" could not be activated because it does not exist.
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.1.4:compile (scala-compile-first) on project zeppelin-flink: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.1.4:compile failed: Plugin net.alchim31.maven:scala-maven-plugin:3.1.4 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven:maven-core:jar:3.0.4, org.apache.maven:maven-settings:jar:3.0.4, org.apache.maven:maven-settings-builder:jar:3.0.4, org.apache.maven:maven-repository-metadata:jar:3.0.4, org.apache.maven:maven-plugin-api:jar:3.0.4, org.apache.maven:maven-model-builder:jar:3.0.4, org.apache.maven:maven-aether-provider:jar:3.0.4, org.sonatype.aether:aether-spi:jar:1.13.1, org.sonatype.aether:aether-impl:jar:1.13.1, org.sonatype.aether:aether-api:jar:1.13.1, org.sonatype.sisu:sisu-inject-plexus:jar:2.3.0, org.apache.maven:maven-model:jar:3.0.4: Could not transfer artifact org.apache.maven:maven-core:jar:3.0.4 from/to central (https://repo.maven.apache.org/maven2): repo.maven.apache.org: Unknown host repo.maven.apache.org -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :zeppelin-flink
No idea where to go from here. Are there any changes required in any of the settings in zeppelin folder ? Suggestions please.
I don't know the reason behind it but this problem got solved. I just ran the below command in the morning and build was successful
mvn install -DskipTests
Not sure if there is a logic behind that but if someone knows please share that.
If you have cloned zeppelin from git, then you need to go to the 'master' folder(in command line) and then run the following command :
mvn clean package -DskipTests -Pspark-1.5.2 -Phadoop-2.7.1

Apache camel file component not moving files

I'm using apache camel 2.12.1 to create a route and then move some files in my local directory, the exmple runs fine but the files are never moved, this is the code for the class.
public class MoveFilesTest {
private static final Log LOG = LogFactory.getLog(MoveFilesTest.class);
public static void main(String args[]) throws Exception {
LOG.debug("create CamelContext");
CamelContext context = new DefaultCamelContext();
// add our route to the CamelContext
context.addRoutes(new RouteBuilder() {
File file = null;
public void configure() {
from("file:data/inbox?delay=100&noop=true")
.process( new Processor() {
public void process(Exchange msg) throws Exception {
File file = msg.getIn().getBody(File.class);
LOG.debug("Processing file: " + file.getName());
}
})
.to("file:data/outbox").end();
}
});
LOG.debug("start the route and let it do its work");
context.start();
context.stop();
}
}
as a note, this code just to work, now i'm working on mac os x 10.7, this is the debug log. i added the noop=false and the delete=true, but the result is the same. thank you
DEBUG [main] (MoveFilesTest.java:24) - create CamelContext
DEBUG [main] (MoveFilesTest.java:45) - start the route and let it do its work
INFO [main] (DefaultCamelContext.java:1498) - Apache Camel 2.12.1 (CamelContext: camel-1) is starting
INFO [main] (ManagedManagementStrategy.java:187) - JMX is enabled
INFO [main] (DefaultTypeConverter.java:50) - Loaded 176 type converters
INFO [main] (DefaultCamelContext.java:1689) - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
INFO [main] (FileEndpoint.java:83) - Endpoint is configured with noop=true so forcing endpoint to be idempotent as well
INFO [main] (FileEndpoint.java:89) - Using default memory based idempotent repository with cache max size: 1000
INFO [main] (DefaultCamelContext.java:2183) - Route: route1 started and consuming from: Endpoint[file://data/inbox?delay=100&noop=true]
INFO [main] (DefaultCamelContext.java:1533) - Total 1 routes, of which 1 is started.
INFO [main] (DefaultCamelContext.java:1534) - Apache Camel 2.12.1 (CamelContext: camel-1) started in 8.936 seconds
INFO [main] (DefaultCamelContext.java:1706) - Apache Camel 2.12.1 (CamelContext: camel-1) is shutting down
INFO [main] (DefaultShutdownStrategy.java:172) - Starting to graceful shutdown 1 routes (timeout 300 seconds)
INFO [Camel (camel-1) thread #2 - ShutdownTask] (DefaultShutdownStrategy.java:600) - Route: route1 shutdown complete, was consuming from: Endpoint[file://data/inbox?delay=100&noop=true]
INFO [main] (DefaultShutdownStrategy.java:217) - Graceful shutdown of 1 routes completed in 0 seconds
INFO [main] (DefaultCamelContext.java:1780) - Apache Camel 2.12.1 (CamelContext: camel-1) uptime 8.953 seconds
INFO [main] (DefaultCamelContext.java:1781) - Apache Camel 2.12.1 (CamelContext: camel-1) is shutdown in 0.013 seconds
Yes, you start Camel and stop it immediately. So, when you put a file to a folder. It wont process cuz camel is already stopped.
Camel contains Main implementation to keep Camel running in standalone application.
There is link: http://camel.apache.org/running-camel-standalone-and-have-it-keep-running.html
CamelContext.start does not block, so basically you are starting the context and then immediately stopping it. You need to wait or block on something until the context should stop. You can reference this thread for some ways of doing this.
I had a similar problem but that was to do with streamCaching() not getting set properly and so the code below was failing
context.getStreamCachingStrategy().setSpoolDirectory(spoolDirectory);
from(uri.toString()).streamCaching().to(destination);
I set the Streaming directly on CamelContext and that solved the problem
CamelContext context = getContext();
context.setStreamCaching(true);
context.getStreamCachingStrategy().setSpoolDirectory(localSpoolDirectory);
context.getStreamCachingStrategy().setSpoolThreshold(Long.parseLong(spoolThreshold.trim()));
context.getStreamCachingStrategy().setBufferSize(Integer.parseInt(bufferSize.trim()));

Resources