How to use Apache Camel to get web page HTML - apache-camel

Question

I want to do nothing more than use Camel to get the HTML of a web page and save it as a local text file. I have tried the below but it doesn't work - it never hits the process code and no file shows up. This seems like it should be fairly simple. What am I doing wrong?
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure()
{
from("jetty:https://2001:4998:124:1507::f001") // www.yahoo.com IP address
.process(new Processor() {
public void process(Exchange exchange) throws Exception
{
System.out.println( "Name="+ exchange.getIn().getBody() );
}
})
.to("file:C:\\temp\\folder2?fileName=web.txt");
}
});
context.start();
Thread.sleep(5000);
context.stop();
System.out.println( "Done" );

Answer 1

from("jetty:https://....") configures the Jetty consumer, which acts as a web server.
You should use a producer, E.g to("jetty:https://..."), to make client requests. Things can be simplified a bit by using the camel-http component instead of Jetty by adding a dependency like:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-http</artifactId>
<version>your-camel-vesion-here</version>
</dependency>
Then the route could look like this. It uses the timer component to initiate the HTTP call:
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
from("timer:getHtml?repeatCount=1")
.to("https://www.yahoo.com")
.to("file:C:\\temp\\folder2?fileName=web.txt");
}
});
If you're using Camel 3.x then you'll also need to add a dependency for camel-timer to make the example work.