I am trying to build Apache Flink from the source. I am strictly following
the instructions given #github flink page. https://github.com/apache/flink
I am facing the error for resolving the dependencies for project
flink-mapr-fs.
Firstly, I tried using latest maven, and afterward, as per instruction at
the flink github page https://github.com/apache/flink
https://github.com/apache/flink , I also tried via maven 3.1.1, but the
same error.
In addition, I tried for flink 1.8.0 and 1.7.2 as well but the same problem exist. The detail of error is as below:
[ERROR] Failed to execute goal on project flink-mapr-fs: Could not resolve dependencies for project org.apache.flink:flink-mapr-fs:jar:1.7.2: Could not transfer artifact com.mapr.hadoop:maprfs:jar:5.2.1-mapr from/to mapr-releases (http://repository.mapr.com/maven/): GET request of: com/mapr/hadoop/maprfs/5.2.1-mapr/maprfs-5.2.1-mapr.jar from mapr-releases failed: Premature end of Content-Length delimited message body (expected: 49411916; received: 39845888 -> [Help 1]
Related
I am trying to update my tool chain from Java 8 to 11.
Doing so I run into issues compiling WSDLs to java using the Apache CXF maven plugin.
I tried with the latest two available versions:
org.apache.cxf:cxf-codegen-plugin:3.3.12 and
org.apache.cxf:cxf-codegen-plugin:3.4.5
When I run the wsdl2java goal (phase generate-sources) it first seems to read all the WSDLs and XSDs ok but at the end it invariably ends in compile errors (see below). So it seems as if it generates code that can't be compiled under Java 11!?!
What dependencies do I need to add? Or what other steps are necessary to get this working again?
Error:
...
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /D:/Projects/KStA_ZH_ZHQuest/code/application/zhquest-sam-web-service/target/generated-sources/cxf/ch/adnovum/nevisidm/ws/services/v1_37/AdminServicePortImpl.java:[10,17] package javax.jws does not exist
...
[ERROR] /D:/Projects/KStA_ZH_ZHQuest/code/application/zhquest-sam-web-service/target/generated-sources/cxf/ch/adnovum/nevisidm/ws/services/v1_37/AdminServicePortImpl.java:[14,22] package javax.jws.soap does not exist
[ERROR] /D:/Projects/KStA_ZH_ZHQuest/code/application/zhquest-sam-web-service/target/generated-sources/cxf/ch/adnovum/nevisidm/ws/services/v1_37/AdminServicePortImpl.java:[15,33] package javax.xml.bind.annotation does not exist
Note: I am aware of these threads: CXF codegen maven plugin doesn't work OpenJDK 11 and What dependencies do I need to run Apache CXF with Spring Boot on Java 11? but the dependencies suggested in either did not fix my issue.
just trying to migrate from flink 1.3 into 1.4 and getting this exception on
linux machine:
(not reproducing at windows).
i've import this package also:
// https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-hadoop2
compile group: 'org.apache.flink', name: 'flink-shaded-hadoop2', version: '1.4.0'
any help?
at flink console:
TriggerWindow(TumblingProcessingTimeWindows(10000), ReducingStateDescriptor{serializer=org.apache.flink.api.java.typeutils.runtime.TupleSerializer#cb6c5dba, reduceFunction=com.clicktale.reducers.MetricsReducer#4e406694}, ProcessingTimeTrigger(), WindowedStream.reduce(WindowedStream.java:241)) -> Sink: Unnamed (1/1)
java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.fs.LocalFileSystem not a subtype
at java.util.ServiceLoader.fail(ServiceLoader.java:239)
at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2364)
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2375)
at org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:99)
at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:401)
at org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.createHadoopFileSystem(BucketingSink.java:1154)
at org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.initFileSystem(BucketingSink.java:411)
at org.apache.flink.streaming.connectors.fs.bucketing.BucketingSink.initializeState(BucketingSink.java:355)
at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.tryRestoreFunction(StreamingFunctionUtils.java:178)
at org.apache.flink.streaming.util.functions.StreamingFunctionUtils.restoreFunctionState(StreamingFunctionUtils.java:160)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.initializeState(AbstractUdfStreamOperator.java:96)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:259)
at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:694)
at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:682)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
at java.lang.Thread.run(Thread.java:748)
I faced a similar (not specifically this, but dependencies related) issues migrating from 1.3 to 1.4.
In my case, I had to re-generate a fresh POM file using maven archetype and then add the needed dependencies one by one.
See Java Quickstart or Scala Quickstart.
Reason being that there has been a major rework on dependency structure. See Release notes for more information.
Note that Flink 1.4 will load any Hadoop jars found via the "hadoop classpath" shell command, and these will be first on the classpath. So if you have an incompatible version of Hadoop installed that the "hadoop" command points at, you can run into this kind of problem.
I have just started learning Apache Flink and found the guide link to start the development in EClipse IDE.
I followed the this to start off but getting the below error
00:20:26,993 INFO org.apache.flink.api.java.ExecutionEnvironment - The job has 0 registered types and 0 default Kryo serializers
Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/Traversable
at java.lang.ClassLoader.defineClass1(Native Method)
Here I have place the Error log log File ... Please let me know if your require more details. Thanks, Nyamath
java.util.zip.ZipException: invalid LOC header (bad signature)
Your Scala jar file provided by Maven seems to be corrupted. Please update your Maven dependencies by executing this from your project folder on the command line:
mvn -U clean install
In Eclipse, right click on your project and click on Update - - > Maven dependencies.
If that does not work you'll need to delete the corrupted Jar file in the .m2/repositories/ folder.
I'm new to scala. I'm trying to get this really interesting example from Matthias Nehlsen to work - github source.
I confirmed that I'm getting twitter data correctly (activator run). However, I'm not getting the generated public assets:
http://hostname:9000/angular/
Gives me this 404:
http://hostname:9000/assets/build/angular/birdwatch.js
http://hostname:9000/#
Gives me a 404 for this:
http://hostname:9000/assets/build/react-js/birdwatch.js
I am running using typesafe-activator 1.2.3 using: "activator run"
BirdWatch% activator run
[info] Loading project definition from ~/git/BirdWatch/project
[info] Set current project to BirdWatch (in build file:~/git/BirdWatch/)
--- (Running the application from SBT, auto-reloading is enabled) ---
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
(Server started, use Ctrl+D to stop and go back to the console...)
[info] a.e.s.Slf4jLogger - Slf4jLogger started
[info] application - Starting new client
On start (forgot to add, after a clean):
Compiling 13 Scala sources and 1 Java source to ~/git/BirdWatch/target/scala-2.10/classes...
[warn] /Users/NathanDunn/git/BirdWatch/conf/routes:6: unreachable code
[warn] GET /cljs-dev/ controllers.BirdWatch.indexCljs
[warn] /Users/NathanDunn/git/BirdWatch/conf/routes:9: unreachable code
[warn] GET /cljs/ controllers.BirdWatch.indexCljsOpt
[warn] two warnings found
I think that this is just indicating that the previous pattern subsumes the latter.
You Need to build the Clients.
Quoting the author :
The different clients are located in the clients/ folder, each of them needs to be built in order to work. Client build artifacts are no longer checked into the git repository in order to avoid bloating the repository size.
Have a look at their read me files:
You need Node.js, Grunt and Bower to build the angular client.
https://github.com/matthiasn/BirdWatch/tree/master/clients/angular
https://github.com/matthiasn/BirdWatch/blob/master/clients/react-js
I am using Eclipse 3.6 (Helios64) and m2eclipse 0.12.1.20110112-1712. I have a new project that only contains two classes that builds a jar. When I right click the project and select "Run As -> Maven package", I get the following error:
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) # switch-provisioning-rest-client --- [INFO] Compiling 2 source files to C:\Devel\EclipseProjects\MyProject\target\classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] Failure executing javac, but could not parse the error:
javac: invalid flag: -s
...
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
When I set up a run configuration with the Goals "package -X", the project builds without error. If I run a configuration with the goal set to compile, it builds without error and I can then run "Maven package" successfully.
This is the only project I'm having issues with. All my other projects build sucessfully when doing a "Maven package".
There's nothing exceptional about the project pom.xml. It refers to the same parent pom as the other projects and contains its project specific dependencies.
Has anybody got any ideas what could be different about this project?
TIA!
This issue was answered on the m2eclipse mailing list:
http://dev.eclipse.org/mhonarc/lists/m2e-users/msg01818.html
Although my pom had 1.6 as the version, the project configuration hadn't been updated.
From: Benjamin Bentmann
This issue is not really specific to m2e but applies to mvn in general, as such the Maven user list might provide a better audience for help.
IIRC, the specific error means you trying to use a JDK 1.5 while you need JDK 1.6+ where the javac option in question was added. Assuming your project should actually compile against JDK 1.5, downgrading the maven-compiler-plugin version might also do, I would expect it started to use the -s option when the annotation processing support was added.