I want build zeppelin 0.8.0 with spark 2.3.2 inbuilt and run it against the same version of spark running not locally without setting SPARK_HOME so that I do not require to have a SPARK installation in the zeppelin node. I have tried the build options given in the document https://zeppelin.apache.org/docs/0.8.0/setup/basics/how_to_build.html but it fails the build trying to build for scala 2.10.
Is this even possible with the 0.8.0 version?
So you can build the 0.8.1-snapshot with spark 2.3.2. But however you cannot connect to an external spark node using the inbuilt spark binaries without giving a seperate spark installation using SPARK_HOME.
You can merge code from this pull request and then the spark-2.3 profile will build with spark 2.3.2 and won't fail with scala 2.10 error.
Or wait for 0.8.1 (first candidate was tagged today!)
Related
I'm trying to start with the DataStream API, but have a missing module.
Any idea what's wrong ?
Version
Python 3.7.9
python -m pip install apache-flink
Code
from pyflink.common.serialization import SimpleStringEncoder
Error
ModuleNotFoundError: No module named pyflink.common.serialization
The DataStream API is not supported yet in PyFlink. This is planned for Flink 1.12, with the initial work described in FLIP-130.
Sorry if you were misled by the master version of the documentation. If you still want to try the Python DataStream API, you can build PyFlink from source.
I am trying to run a flink steaming program that uses kafka connector(latest universal connector).
The jobs runs without any problem on IntelliJ but when I am submitting the code build into jar using sbt package is giving me below error.
java.lang.ClassNotFoundException: org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase
I also used the jar built using traditional IntellIJ option but still i get the above error.
Most probably the issue is the fact that You are not including the dependencies in Your JAR file. Connector dependencies are not included in the Flink binary.
Generally, the preferred way of tackling this issue is to use the proper plugin for Your build tool like shade-plugin for Maven or assembly for sbt to create so-called fat-jar i.e. the JAR with the dependencies included.
I'm using Jenkins ver 2.121.2. I'm trying to configure the Groovy plugin to run groovy scripts via a Jenkins job. The plugin documentation provided here
does not appear to match the UI I see in the System Configuration - Groovy section of this version of Jenkins. The various parameters don't match and so far I'm unable to get the plugin to run a simply groovy script. Not being familar with java and how the classpath works I'm not able to loosely interpret the instructions and get it working. If anyone can point me at documentation that more closely matches the Groovy plugin with the most current version of Jenkins I would appreciate it.
First, you need to configure an instance (or instances) of Groovy to run scripts in your builds. You do that on the Global Tool Configuration page.
You should have a least one JDK configured in the Global Tools Configuration (or have JAVA_HOME defined).
You can have Jenkins install the version of Groovy you want (on first use), or you can install Groovy yourself and point to it as in the example below:
Once this is in place, you can use the Execute Groovy Script build step in a build:
In our application we have encountered impotency issue which is fixed by https://issues.apache.org/jira/browse/CAMEL-9480. we are currently using service mix 5.5.0 with following dependencies.
Activiti 5.17.0
Apache ActiveMQ 5.11.1
Apache Camel 2.15.2
Apache CXF 3.0.4
Apache Karaf 2.4.3 .
So in-order to get fix for CAMEL-9480, i tried to upgrade camel to 2.16.2 by doing service mix upgrade to 5.6.0 which supports following dependencies
Activiti 5.19.0.2
Apache ActiveMQ 5.12.3
Apache Camel 2.16.2
Apache CXF 3.1.4
Apache Karaf 2.4.4
after upgrade i am getting following error during deployment
[caused by: Unable to resolve 295.0: missing requirement [295.0] osgi.wiring.package; (&(osgi.wiring.package=org.eclipse.jetty.util.log)(version>=9.2.0)(!(version>=10.0.0)))]
when i checked camel-jetty-2.16.2.pom,it uses camel-jetty9 and in karaf features its mentioned as below.
<details>camel-jetty9 intend to work with jetty9, so this feature only works in the karaf container which support jetty9, e.g. karaf 4.x</details>
so is it possible to use camel-2.16.2 with karaf 2.4.4 or we need upgrade to karaf 4.x?
There are two bundles and two features:
camel-jetty which support jetty8 (and Karaf 2/3)
camel-jetty9 which support jetty9 and Karaf 4
However, It depends too on the other camel features you use. Some feature uses a feature jetty without versions, other use directly camel-jetty9 or camel-jetty, etc. Sometimes, It necessary to rewrite some feature to fix a version range, but in most case, it works out of the box.
I'm trying to use Google Cloud Endpoints with Android client, built by Maven.
The problem is that appengine-maven-plugin v1.8.3+ require Maven v3.1.0+
[ERROR] Failed to execute goal com.google.appengine:appengine-maven- plugin:1.8.4:endpoints_get_discovery_doc (default) on project pro-gae: The plugin com.google.appengine:appengine-maven-plugin:1.8.4 requires Maven version 3.1.0 -> [Help 1]
while android-maven-plugin v3.6.1 doesn't (and will not) work with Maven v3.1.0 (see devs discussion at https://code.google.com/p/maven-android-plugin/issues/detail?id=395). The error
A required class was missing while executing com.jayway.maven.plugins.android.generation2:android-maven-plugin:3.6.1:generate-sources: Lorg/sonatype/aether/RepositorySystem;
So I'm kinda locked out. One plugin asks for v3.1.0+, while the other one claims Maven 3.1.0 to be fatally buggy. So I left out to generate Endpoints client sources manually.
Any other ideas, besides waiting for fixed Maven 3.1.1? It doesn't seem to happen soon, because Maven 3.1.0 was released more than two months ago.
The latest version of the Android Maven Plugin is 3.8.0 and it now requires Maven 3.1.1+. This should allow you to use the latest appengine plugin as well. And Maven 3.1.1 has been released in the meantime as well.
Update Sept 2015: Latest version of the plugin is 4.3.0 with 4.3.1 around the corner. The 4x branch works with Maven 3.0.5+ but we strongly encourage users to use the latest Maven release.
Hmm, looks resolved by just using older version (1.8.2) of appengine-maven-plugin.
One other way of building is to configure the Run Configuration to use an external Maven Runtime.
To do this in Eclipse for Windows :
Right Click Project -> Run Configuration -> Create a new configuration -> In Main tab configure Maven Runtime.
I hope this helps.