I am trying to run this
example where some DataStreams are being merged. I am using Eclipse for this and I added the Maven dependencies as specified here. Here are my dependencies:
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-java</artifactId>
<version>1.0.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.10</artifactId>
<version>1.0.3</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_2.10</artifactId>
<version>1.0.3</version>
</dependency>
<!-- http://mvnrepository.com/artifact/org.apache.commons/commons-collections4 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.0</version>
</dependency>
</dependencies>
However, some classes are not being recognized (See picture).
The example you're linking to is written for Flink 0.9 and you're adding dependencies for Flink 1.0. The DataStream API has been heavily re-worked between these versions. You should either use dependencies for Flink 0.9.x or try a more recent example. For instance, you can try out the WikipediaAnalysis example from the Flink v1.0 documentation.
Related
I tried looking for solutions to my problem but I haven't found any help.
Our application is using Redis Lettuce but during deployment using Payara 5.2022.3, I can't proceed and it's because of the LettuceCdiExtension error. Can anyone help me? I'm also using jdk11 if it helps
org.glassfish.deployment.common.DeploymentException: CDI deployment failure:Error instantiating :io.lettuce.core.support.LettuceCdiExtension -- Error instantiating :io.lettuce.core.support.LettuceCdiExtension
at org.jboss.weld.util.ServiceLoader.createInstance(ServiceLoader.java:315)
at org.jboss.weld.util.ServiceLoader.prepareInstance(ServiceLoader.java:247)
at org.jboss.weld.util.ServiceLoader.loadService(ServiceLoader.java:215)
at org.jboss.weld.util.ServiceLoader.loadServiceFile(ServiceLoader.java:185)
at org.jboss.weld.util.ServiceLoader.reload(ServiceLoader.java:165)
at org.jboss.weld.util.ServiceLoader.iterator(ServiceLoader.java:289)
at org.glassfish.weld.DeploymentImpl.getExtensions(DeploymentImpl.java:495)
at org.glassfish.weld.WeldDeployer.processApplicationLoaded(WeldDeployer.java:518)
at org.glassfish.weld.WeldDeployer.event(WeldDeployer.java:434)
at org.glassfish.kernel.event.EventsImpl.send(EventsImpl.java:131)
Exception while loading the app : CDI deployment failure:Error instantiating :io.lettuce.core.support.LettuceCdiExtension -- Error instantiating :io.lettuce.core.support.LettuceCdiExtension]]
This is part of the pom.xml
<!-- JPA -->
<dependency>
<groupId>javax.transaction</groupId>
<artifactId>jta</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-envers</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
</dependency>
<dependency>
<groupId>io.lettuce</groupId>
<artifactId>lettuce-core</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aspects</artifactId>
</dependency>
<!-- JPA End -->
I try to check out Apache Flink's function to report metrics to external systems. I started with the JMX reporter because it doesn't need an additional dependency since it's included in Flink via "org.apache.flink.metrics.jmx.JMXReporter". I'm running everything from IntelliJ, so that I had to add the setup to my configuration like this:
Properties props = new Properties();
props.put("metrics.reporter.jmx.factory.class", "org.apache.flink.metrics.jmx.JMXReporterFactory");
Configuration conf = ConfigurationUtils.createConfiguration(props);
env = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(conf);
So far so good. I got this warning from my console:
14:06:58,715 WARN org.apache.flink.runtime.metrics.ReporterSetup [] - The reporter factory (org.apache.flink.metrics.jmx.JMXReporterFactory) could not be found for reporter jmx. Available factories: [].
So it tells me there's no available reporter factory, right? But why? In addition I get following error message, if I try to add the reporter setup with
conf.setClass("metrics.reporter.jmx.factory.class", org.apache.flink.metrics.jmx.JMXReporterFactory);
Cannot resolve symbol 'jmx'
So it can't find jmx what is strange cause the documentary told me it's included.
This is my pom.xml:
<dependencies>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-walkthrough-common_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-runtime-web_2.12</artifactId>
<version>${flink.version}</version>
</dependency>
<!-- This dependency is provided, because it should not be packaged into the JAR file. -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
<scope>provided</scope>
</dependency>
<!-- Add connector dependencies here. They must be in the default scope (compile). -->
<!-- Example:
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-kafka_${scala.binary.version}</artifactId>
<version>${flink.version}</version>
</dependency>
-->
<!-- Add logging framework, to produce console output when running in the IDE. -->
<!-- These dependencies are excluded from the application JAR by default. -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>${log4j.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>${log4j.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>${log4j.version}</version>
<scope>runtime</scope>
</dependency>
</dependencies>
I am trying to set up a hystrix-dashboard with turbine. I am not using Eureka and want to use the FileBasedInstanceDiscovery. I am trying to configure this but following the online documentation doesn't seem to work. It always tries to use the Eureka discovery client. I tried excluding the eureka from my pom but then it falls back to another discovery client CommonsInstanceDiscovery
Here is my application.properties:
turbine.aggregator.clusterConf=mycluster
turbine.instanceUrlSuffix.mycluster=8080/hystrix.stream
turbine.FileBasedInstanceDiscovery.filePath=turbine.hostnames.txt
turbine.InstanceMonitor.eventStream.skipLineLogic.enabled=false
InstanceDiscovery.impl=com.netflix.turbine.discovery.FileBasedInstanceDiscovery
Is there another way to do this? Also, for the filePath for my turbine hostnames, where does it start looking? Can I have the file under my resources directory of my jar?
here is my pom file dependencies:
<properties>
<java.version>1.8</java.version>
<spring-cloud.version>Greenwich.SR1</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web-services</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-hystrix-dashboard</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-turbine</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Some of the documentation is unclear and conflicting for turbine, I think this is because there is the spring-cloud-netflix project and a standalone turbine project. At any rate, if you are building a spring-boot app then this property is not useful:
InstanceDiscovery.impl=com.netflix.turbine.discovery.FileBasedInstanceDiscovery
If you want to change the implementation of InstanceDiscovery it is pretty simple, just create a bean that implements InstanceDiscovery like this:
#Bean
public InstanceDiscovery instanceDiscovery() {
//choose either one of the provided implementations from spring or
//create your own
return new ConfigPropertyBasedDiscovery();
//return new FileBasedInstanceDiscovery();
}
Make sure you put this in a #Configuration class. I originally had mine just in my SpringBootApplication class but this doesn't allow you to override the default implementation.
I am using Apache Shiro with Guice on Google App Engine.
Following filter chain is present in configureShiroWeb() function
addFilterChain("/**/first/second/third/**", AUTHC_BASIC, config(REST, "X"));
addFilterChain("/**/first/**", AUTHC_BASIC, config(REST, "Y"));
When a request is made for an API- example.appspot.com/v1/first/second/third, the first filter is bypassed and the access is granted for a user with permission Y and not with X.
I am using the following shiro and guice related dependencies:
<dependency>
<groupId>org.apache.shiro</groupId>
<artifactId>shiro-core</artifactId>
<version>1.2.4</version>
</dependency>
<dependency>
<groupId>org.apache.shiro</groupId>
<artifactId>shiro-web</artifactId>
<version>1.2.4</version>
</dependency>
<dependency>
<groupId>org.apache.shiro</groupId>
<artifactId>shiro-guice</artifactId>
<version>1.2.4</version>
</dependency>
<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
<version>3.0</version>
</dependency>
<dependency>
<groupId>com.google.inject.extensions</groupId>
<artifactId>guice-servlet</artifactId>
<version>3.0</version>
</dependency>
<dependency>
<groupId>com.google.inject.extensions</groupId>
<artifactId>guice-multibindings</artifactId>
<version>3.0</version>
</dependency>
<dependency>
<groupId>com.sun.jersey.contribs</groupId>
<artifactId>jersey-guice</artifactId>
<version>1.8</version>
</dependency>
Upgrade Shiro, If you are still running into this issue let us know.
I am trying to run Apache Flink but getting the error as dependency not available.
Is the module flink-streaming-java still available.
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>0.10-SNAPSHOT</version>
<scope>test</scope>
<type>test-jar</type>
</dependency>
I have used 0.9.1 and few more versions, but none of them worked.
On checking further in flink-streaming-java https://github.com/apache/flink/blob/master/flink-streaming-java/pom.xml
found -
<!-- disable fork reuse for the streaming project, because of
incorrect declaration of tests -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<reuseForks>false</reuseForks>
</configuration>
</plugin>
Flink Streaming Java ยป 0.10.0
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>0.10.0</version>
</dependency>
https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-java
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.11</artifactId>
<version>1.4.2</version>
<scope>provided</scope>
</dependency>