I try different things to create a standalone camel using the cdi.main
org.apache.camel.cdi.Main.main();
//not org.apache.camel.Main
I have also configured deltaspike and I see the cdi container, but my CamelContext is not started.
If someone already runned a Camel boot cdi standalone, could you send the code ?
update:
I keep on getting this kind of error:
org.apache.webbeans.exception.WebBeansDeploymentException: javax.enterprise.inject.AmbiguousResolutionException: There is more than one Bean with type org.apache.camel.CamelContextQualifiers: [#javax.enterprise.inject.Default()]
for injection into Field Injection Point, field name : ctx, Bean Owner : [ContextInitializer, WebBeansType:MANAGED, Name:null, API Types:[java.lang.Object,proj.core.ContextInitializer], Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any]]
found beans:
CdiCamelContext, WebBeansType:THIRDPARTY, Name:CamelContext, API Types:[org.apache.camel.CamelContext,org.apache.camel.cdi.CdiCamelContext,java.lang.Object], Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] from jar:file:/P:/atos/common/apache-maven-repo/org/apache/camel/camel-cdi/2.16.1/camel-cdi-2.16.1.jar!/org/apache/camel/cdi/CdiCamelContext.class
CamelContext, WebBeansType:PRODUCERMETHOD, Name:null, API Types:[org.apache.camel.CamelContext,org.apache.camel.SuspendableService,org.apache.camel.RuntimeConfiguration,java.lang.Object,org.apache.camel.Service], Qualifiers:[javax.enterprise.inject.Default,javax.enterprise.inject.Any] from file:/P:/atos/proj-vnext/proj-core/target/classes/proj/core/CamelContextFactory.class
Just if someone encounters the same problem, I just found out the problem and it was just a configuration problem in the pom.xml. mixing camel core and camel cdi version
here is my working config => camel, OpenWebBeans container and deltaspike
*/ code that boots the app
org.apache.camel.cdi.Main maincdi = new org.apache.camel.cdi.Main(){};
maincdi.run();
*/ camel context producer
public class CamelContextFactory {
#Produces
#ApplicationScoped
CamelContext customize() {
DefaultCamelContext context = new DefaultCamelContext();
context.setName("my-custom-camel-context");
return context;
}
void cleanUp(#Disposes CamelContext context) {
// ...
}
}
*/create a context initializer
#ApplicationScoped
public class ContextInitializer {
#Inject
private CamelContext ctx;
#Inject
#Any
private Instance<RouteBuilder> routes;
#PostConstruct
public void init() {
// add routes
for (RouteBuilder route : routes) {
try {
ctx.addRoutes(route);
} catch (Exception ex) {
Logger.getLogger(ContextInitializer.class.getName()).log(Level.SEVERE, null, ex);
}
}
//this.ctx.start();
}
#PreDestroy
public void stop() {
//this.ctx.stop();
}
}
*/ A route class that will be injected in 'routes' property
public class SomeRouteClass extends RouteBuilder {
#Override
public void configure() throws Exception {
from("timer:foo?period=2000")
//.bean(testBean)
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
System.out.println("route called from context " + exchange.getContext().getName());
}
});
}
}
*/ part of the pom.xml
<properties>
<deltaspike.version>1.7.2</deltaspike.version>
<!--<weld.version>2.3.3.Final</weld.version>-->
<owb.version>1.6.3</owb.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.apache.deltaspike.distribution</groupId>
<artifactId>distributions-bom</artifactId>
<version>${deltaspike.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
<version>2.18.1</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-cdi</artifactId>
<version>2.18.1</version>
</dependency>
<!--delta spike core-->
<dependency>
<groupId>org.apache.deltaspike.core</groupId>
<artifactId>deltaspike-core-api</artifactId>
<version>${deltaspike.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.deltaspike.core</groupId>
<artifactId>deltaspike-core-impl</artifactId>
<version>${deltaspike.version}</version>
<scope>runtime</scope>
</dependency>
<!--container control-->
<dependency>
<groupId>org.apache.deltaspike.cdictrl</groupId>
<artifactId>deltaspike-cdictrl-api</artifactId>
<version>${deltaspike.version}</version>
<scope>compile</scope>
</dependency>
<!--open web bean dependency-->
<dependency>
<groupId>org.apache.openwebbeans</groupId>
<artifactId>openwebbeans-impl</artifactId>
<version>${owb.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.openwebbeans</groupId>
<artifactId>openwebbeans-spi</artifactId>
<version>${owb.version}</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-jcdi_1.1_spec</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-atinject_1.0_spec</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-interceptor_1.2_spec</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.apache.geronimo.specs</groupId>
<artifactId>geronimo-annotation_1.2_spec</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>org.apache.deltaspike.cdictrl</groupId>
<artifactId>deltaspike-cdictrl-owb</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>javax.enterprise</groupId>
<artifactId>cdi-api</artifactId>
<version>1.2</version>
</dependency>
</dependencies>
=> My mistake was that I had camel-core version 2.18.1 and camel cdi 2.16.1
Related
I'm trying to connect my Springboot aplication to MongoDb. When i ran my application and i got this error:
Description:
Field userRepo in com.universitem.service.Impl.UserDetailsServiceImpl required a bean of type 'com.universitem.repository.UserRepository' that could not be found.
Action:
Consider defining a bean of type 'com.universitem.repository.UserRepository' in your configuration.
application.properties file:
spring.jpa.hibernate.ddl-auto=update
spring.data.mongodb.uri= /includes the uri
spring.data.mongodb.database=test
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
spring.jpa.properties.hibernate.globally_quoted_identifiers=true
universitem.app.secret=UniversityToken
universitem.expires.in=120000
refresh.token.expires.in=604800
Main file :
#SpringBootApplication(exclude = {SecurityAutoConfiguration.class})
#EnableJpaRepositories(basePackages = {"com.universitem.repository"})
public class UniversitemApplication {
public static void main(String[] args) {
SpringApplication.run(UniversitemApplication.class, args);
}
}
Security Configuration file:
#Configuration
#EnableWebSecurity
#Data
public class SecurityConfig {
private UserDetailsServiceImpl userDetailsService;
private JwtAuthenticationEntryPoint handler;
public SecurityConfig(UserDetailsServiceImpl userDetailsService, JwtAuthenticationEntryPoint handler) {
this.userDetailsService = userDetailsService;
this.handler = handler;
}
#Bean
public JwtAuthenticationFilter jwtAuthenticationFilter() {
return new JwtAuthenticationFilter();
}
#Bean
public PasswordEncoder passwordEncoder() {
return new BCryptPasswordEncoder();
}
#Bean
public AuthenticationManager authenticationManager(AuthenticationConfiguration authenticationConfiguration) throws Exception {
return authenticationConfiguration.getAuthenticationManager();
}
#Bean
public CorsFilter corsFilter() {
UrlBasedCorsConfigurationSource source = new UrlBasedCorsConfigurationSource();
CorsConfiguration config = new CorsConfiguration();
config.setAllowCredentials(true);
config.addAllowedOrigin("*");
config.addAllowedHeader("*");
config.addAllowedMethod("OPTIONS");
config.addAllowedMethod("HEAD");
config.addAllowedMethod("GET");
config.addAllowedMethod("PUT");
config.addAllowedMethod("POST");
config.addAllowedMethod("DELETE");
config.addAllowedMethod("PATCH");
source.registerCorsConfiguration("/**", config);
return new CorsFilter(source);
}
#Bean
public SecurityFilterChain filterChain(HttpSecurity httpSecurity) throws Exception {
httpSecurity
.cors()
.and().csrf().disable()
.exceptionHandling().authenticationEntryPoint(handler).and()
.sessionManagement().sessionCreationPolicy(SessionCreationPolicy.STATELESS).and()
.authorizeRequests()
.antMatchers(HttpMethod.GET, "/comment/getAllUniversityComment{universityId}")
.permitAll()
.antMatchers(HttpMethod.GET, "/university/getAll")
.permitAll()
.antMatchers(HttpMethod.POST, "/userType/add")
.permitAll()
.antMatchers(HttpMethod.POST, "/universityType/add")
.permitAll()
.antMatchers(HttpMethod.GET, "/userType/getAll")
.permitAll()
.antMatchers("/userAuth/**")
.permitAll()
.anyRequest().authenticated();
httpSecurity.addFilterBefore(jwtAuthenticationFilter(), UsernamePasswordAuthenticationFilter.class);
return httpSecurity.build();
}
}
User Repository file:
#Repository
public interface UserRepository extends JpaRepository<User,Long> {
User findByUserName(String userName);
User findByUserMail(String userMail);
}
User Details Service Implementation file:
#Service
public class UserDetailsServiceImpl implements UserDetailsService {
#Autowired
private UserRepository userRepo;
#Override
public UserDetails loadUserByUsername(String userMail) throws UsernameNotFoundException {
User user =userRepo.findByUserMail(userMail);
return JwtUserDetails.create(user);
}
public UserDetails loadUserById(Long userId) {
User user = userRepo
.findById(userId).get();
return JwtUserDetails.create(user);
}
}
pom.xml file:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.jsonwebtoken</groupId>
<artifactId>jjwt</artifactId>
<version>0.9.1</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
I tried to add scanBasePackages to #SpringBootApplication like below. Actually database has been started but when i tried to sent request i got 404 Not Found error on postman.
#SpringBootApplication(scanBasePackages = {"com.universitem.repository"}, exclude = {SecurityAutoConfiguration.class})
I am confused how to implement CamelCorrelationID/JMSCorrelationID in a way where each application listen to its own response from server using messaging system. The queues are shared in this scenario. I have implemented my own CorrelationID to maintain the state of application. here I am using two client applications (Client-A and Client-B) sending/receiving the request to/from server
Currently both applications are reading each other responses from server, instead each application should listens to its own response based on CorrelationID passed in the header of request message.
Need help to resolve this.
Below are the code details:
Client-A code:
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.12</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.camel.a</groupId>
<artifactId>CorrelationId-A</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>CorrelationId-A</name>
<description>Understanding CorrelationId</description>
<properties>
<java.version>1.8</java.version>
<camel.version>3.11.0</camel.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-amqp-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
ClientARequestRoute.java
package com.camel.a.route;
import java.util.UUID;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;
#Component
public class ClientARequestRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("timer://runOnce?repeatCount=1")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
UUID uuid = UUID.randomUUID();
String uniqueId = uuid.toString();
String testMsg = "Sending message from Client A with Unique Id as - " + uniqueId;
exchange.getIn().setHeader(Exchange.CORRELATION_ID, uniqueId);
exchange.getIn().setBody(testMsg, String.class);
}
})
.log("${body}")
.to("amqp:queue:RequestQueue");
}
}
ResponseServerRoute.java
package com.camel.a.route;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;
#Component
public class ResponseServerRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("amqp:queue:ResponseQueue")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String uniqueId = (String)exchange.getIn().getHeader(Exchange.CORRELATION_ID);
System.out.println("Received Correlation Id - " + uniqueId);
String msgReceived = (String) exchange.getIn().getBody(String.class);
exchange.getIn().setBody(msgReceived);
}
})
.log("${body}")
.end();
}
}
Client-B code:
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.12</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.camel.b</groupId>
<artifactId>CorrelationId-B</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>CorrelationId-B</name>
<description>Understanding CorrelationId</description>
<properties>
<java.version>1.8</java.version>
<camel.version>3.11.0</camel.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-amqp-starter</artifactId>
<version>${camel.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
ClientBRequestRoute.java
package com.camel.b.route;
import java.util.UUID;
import org.apache.camel.Exchange;
import org.apache.camel.ExchangePattern;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
#Component
public class ClientBRequestRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("timer://runOnce?repeatCount=1")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
UUID uuid = UUID.randomUUID();
String uniqueId = uuid.toString();
String testMsg = "Sending message from Client B with Unique Id as - " + uniqueId;
exchange.getIn().setHeader(Exchange.CORRELATION_ID, uniqueId);
exchange.getIn().setBody(testMsg, String.class);
}
})
.log("${body}")
.to("amqp:queue:RequestQueue");
}
}
ResponseServerRoute.java
package com.camel.b.route;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.springframework.stereotype.Component;
#Component
public class ResponseServerRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("amqp:queue:ResponseQueue")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String uniqueId = (String)exchange.getIn().getHeader(Exchange.CORRELATION_ID);
System.out.println("Received Correlation Id - " + uniqueId);
String msgReceived = (String) exchange.getIn().getBody(String.class);
exchange.getIn().setBody(msgReceived);
}
})
.log("${body}")
.end();
}
}
I'm confused about the best way to configure the SJMS2 component. I'm using camel-spring-boot-starter in a simple test application and trying to write from a ProducerTemplate to ActiveMQ Artemis using the SJMS2 Camel Component. The component documentation says it handles things like connection caching and such which I would normally configure in my ConnectionFactory bean, so I'm getting the sense that I should defining less in my Configuration than I would without using Camel.
The documentation seems to lack a clear example of how to configure an jsms2 route and its ConnectionFactory when using Camel Spring. Is there a good example of this somewhere or could someone show me how that would look? Here's my current test application:
Spring Boot Application:
#SpringBootApplication
public class WebhookpublisherApplication implements CommandLineRunner {
#Autowired
private ProducerTemplate producerTemplate;
public static void main(String[] args) {
SpringApplication.run(WebhookpublisherApplication.class, args);
}
#Override
public void run(String... args) throws Exception {
producerTemplate.sendBody("direct:test", "testBody");
}
}
Route:
#Component
public class RestToJmsRoute extends RouteBuilder {
#Override
public void configure() {
from("direct:test")
.log("Recieved a test body: ${body}")
.to("sjms2:topic:test-topic");
}
}
JMS Config (which is clearly not being used by the Camel component)
#Configuration
public class JmsConfig {
#Bean
public ActiveMQConnectionFactory activeMQConnectionFactory() {
return new ActiveMQConnectionFactory("tcp://localhost:61616");
}
}
POM.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.3.0.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.demo</groupId>
<artifactId>cameljmstest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>cameljmstest</name>
<description>test app</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-artemis</artifactId>
<version>2.3.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>3.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-rest</artifactId>
<version>3.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-sjms2</artifactId>
<version>3.3.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
The above is resulting in the following exception:
Caused by: java.lang.IllegalArgumentException: ConnectionResource or
ConnectionFactory must be configured for sjms2://test-topic
I ended up adding the following configuration class:
#Configuration
public class CamelContextConfig {
#Bean
public CamelContextConfiguration contextConfiguration() {
return new CamelContextConfiguration() {
#Override
public void beforeApplicationStart(CamelContext context) {
Sjms2Component component = new Sjms2Component();
ActiveMQConnectionFactory activeMqConnectionFactory = new ActiveMQConnectionFactory("tcp://myArtemisHost:61616");
activeMqConnectionFactory.setUser("user");
activeMqConnectionFactory.setPassword("pass");
component.setConnectionFactory(activeMqConnectionFactory);
context.addComponent("sjms2", component);
}
#Override
public void afterApplicationStart(CamelContext camelContext) {
}
};
}
}
and then I changed my pom dependencies to these:
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-artemis</artifactId>
<version>2.3.0.RELEASE</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-spring-boot-starter</artifactId>
<version>3.3.0</version>
</dependency>
<dependency>
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-sjms2-starter</artifactId>
<version>3.3.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
When I run the official demo,it's not work
code url: https://ci.apache.org/projects/flink/flink-docs-release-1.5/quickstart/run_example_quickstart.html#writing-a-flink-program
package wikiedits;
import org.apache.flink.api.common.functions.FoldFunction;
import org.apache.flink.api.java.functions.KeySelector;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.datastream.KeyedStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.streaming.connectors.wikiedits.WikipediaEditEvent;
import org.apache.flink.streaming.connectors.wikiedits.WikipediaEditsSource;
public class WikipediaAnalysis {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment see = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<WikipediaEditEvent> edits = see.addSource(new WikipediaEditsSource());
KeyedStream<WikipediaEditEvent, String> keyedEdits = edits
.keyBy(new KeySelector<WikipediaEditEvent, String>() {
#Override
public String getKey(WikipediaEditEvent event) {
return event.getUser();
}
});
DataStream<Tuple2<String, Long>> result = keyedEdits
.timeWindow(Time.seconds(5))
.fold(new Tuple2<>("", 0L), new FoldFunction<WikipediaEditEvent, Tuple2<String, Long>>() {
#Override
public Tuple2<String, Long> fold(Tuple2<String, Long> acc, WikipediaEditEvent event) {
acc.f0 = event.getUser();
acc.f1 += event.getByteDiff();
return acc;
}
});
result.print();
see.execute();
}
}
run command:
$ mvn exec:java -Dexec.mainClass=wikiedits.WikipediaAnalysis
run result:
Exception in thread "main" org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot serialize operator object class org.apache.flink.streaming.runtime.operators.windowing.WindowOperator.
at org.apache.flink.streaming.api.graph.StreamConfig.setStreamOperator(StreamConfig.java:187)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setVertexConfig(StreamingJobGraphGenerator.java:322)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createChain(StreamingJobGraphGenerator.java:216)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createChain(StreamingJobGraphGenerator.java:207)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.setChaining(StreamingJobGraphGenerator.java:177)
at org.apache.flink.streaming.api.graph.StreamingJobGraphGenerator.createJobGraph(StreamingJobGraphGenerator.java:126)
at org.apache.flink.streaming.api.graph.StreamGraph.getJobGraph(StreamGraph.java:580)
at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:89)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1170)
at wikiedits.WikipediaAnalysis.main(WikipediaAnalysis.java:42)
Caused by: java.io.IOException: Unable to serialize default value of type Tuple2.
at org.apache.flink.api.common.state.StateDescriptor.writeObject(StateDescriptor.java:334)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1128)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.apache.flink.util.InstantiationUtil.serializeObject(InstantiationUtil.java:512)
at org.apache.flink.util.InstantiationUtil.writeObjectToConfig(InstantiationUtil.java:442)
at org.apache.flink.streaming.api.graph.StreamConfig.setStreamOperator(StreamConfig.java:184)
... 9 more
Caused by: java.lang.NullPointerException
at org.apache.flink.api.common.state.StateDescriptor.writeObject(StateDescriptor.java:326)
... 25 more
I was able to get this example running from the command line by removing the provided scope on the flink dependencies. Those four dependencies need to exactly match what's shown in the documentation (which obviously needs to be clearer on this point):
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-wikiedits_2.11</artifactId>
<version>${flink.version}</version>
</dependency>
</dependencies>
I wonder if there is any best practice for junit testing of Solr 3.6. I want to automate testing of the Solr schema. Earlier posts mentioned the EmbeddedSolrServer. This class seems to have been abandoned from any version between 1.4 an 3.6. I use Spring 3.0.x and Maven for the project.
The options I considered are:
writing a Junit Test Runner
put the jetty startup code in the #Before or #BeforeClass method
start a solr server in maven (probably not a good option)
put some code in the spring test-context
I've used something similar to what's on this page to run these kinds of tests, all done with EmbeddedSolrServer on Solr 3.4.0. This is a simple approach, but if you want to automate Solr schema testing, it could be enough and isn't hard to implement.
It basically boils down to:
Adding references to junit, solr-core, slf4j-simpleand servlet-apito your pom.xml:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.7</version>
<type>jar</type>
<scope>test</scope>
</dependency>
<!-- dependencies needed for Solr integration test-->
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-core</artifactId>
<version>1.4.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.6.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>test</scope>
</dependency>
And as an example test case, he adds this:
import java.io.IOException;
import org.apache.solr.client.solrj.embedded.EmbeddedSolrServer;
import org.apache.solr.util.AbstractSolrTestCase;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServer;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.common.params.SolrParams;
import org.junit.Before;
import org.junit.Test;
import static org.junit.Assert.assertEquals;
public class SolrSearchConfigTest extends AbstractSolrTestCase {
private SolrServer server;
#Override
public String getSchemaFile() {
return "solr/conf/schema.xml";
}
#Override
public String getSolrConfigFile() {
return "solr/conf/solrconfig.xml";
}
#Before
#Override
public void setUp() throws Exception {
super.setUp();
server = new EmbeddedSolrServer(h.getCoreContainer(), h.getCore().getName());
}
#Test
public void testThatNoResultsAreReturned() throws SolrServerException {
SolrParams params = new SolrQuery("text that is not found");
QueryResponse response = server.query(params);
assertEquals(0L, response.getResults().getNumFound());
}
#Test
public void testThatDocumentIsFound() throws SolrServerException, IOException {
SolrInputDocument document = new SolrInputDocument();
document.addField("id", "1");
document.addField("name", "my name");
server.add(document);
server.commit();
SolrParams params = new SolrQuery("name");
QueryResponse response = server.query(params);
assertEquals(1L, response.getResults().getNumFound());
assertEquals("1", response.getResults().get(0).get("id"));
}
}
edit: I haven't used Solr in quite a while, but I think this could still be a good starting point.
This is my take on it. It doesn't extend AbstractSolrTestCase, just a regular test class.
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertThat;
import java.io.File;
import java.io.IOException;
import org.apache.commons.io.FileUtils;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.embedded.EmbeddedSolrServer;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.client.solrj.response.UpdateResponse;
import org.apache.solr.common.SolrInputDocument;
import org.apache.solr.core.CoreContainer;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
public class EmbeddedSolrServerTest {
private static final int SUCCESS = 0;
private final String indexLocation = "tomcat7/apps/apache-solr-3.6.0";
private EmbeddedSolrServer server;
#Before
public void setUp() throws Exception {
System.setProperty("solr.solr.home", indexLocation);
CoreContainer coreContainer = new CoreContainer.Initializer().initialize();
server = new EmbeddedSolrServer(coreContainer, "collection1");
}
#After
public void tearDown() throws Exception {
server.shutdown();
removeIndexDirectory();
}
#Test
public void testSolrSchema01() throws Exception {
SolrInputDocument doc1 = new SolrInputDocument();
doc1.addField("id", "123");
doc1.addField("something_txt", "super wombat");
UpdateResponse ur = server.add(doc1);
assertThat(ur.getStatus(), is(SUCCESS));
server.commit();
QueryResponse response1 = server.query(new SolrQuery("*:*"));
assertThat(response1.getResults().getNumFound(), is(1L));
QueryResponse response2 = server.query(new SolrQuery("something_txt:*wombat*"));
assertThat(response2.getResults().getNumFound(), is(1L));
}
private void removeIndexDirectory() throws IOException {
File indexDir = new File(indexLocation, "data/index");
FileUtils.deleteDirectory(indexDir);
}
}
My POM has the following dependencies:
<properties>
<solr.version>3.6.0</solr.version>
<httpcomponents.version>4.3.2</httpcomponents.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-solrj</artifactId>
<version>${solr.version}</version>
</dependency>
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-core</artifactId>
<version>${solr.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${httpcomponents.version}</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpmime</artifactId>
<version>${httpcomponents.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-library</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
</dependencies>
I've set up a separate project to keep all my Solr config in and run tests against it. The directory structure is:
solr-test
|--src
| |--test
|--tomcat7
` |--apps
|--apache-solr-3.6.0
|--conf
|--data
|--index
This seems to work well for me. This set up will also let you write tests using HttpSolrServer to test a remote server's schema.
Also, note the server.commit() call in the middle of the test. This needs to be there otherwise the Solr transaction isn't complete and the new document won't be visible.
I'm using https://github.com/moliware/travis-solr for my tests, maybe it's useful for you too.