I am so stuck in this. I have followed each and every single instruction here (https://github.com/apache/incubator-zeppelin)
to install Apache-Zeppelin. I need this for Spark.
Below are various required applications installed in UBUNTU (VM)
hduser#HDPMaster:~$ java -version
java version "1.7.0_91"
OpenJDK Runtime Environment (IcedTea 2.6.3) (7u91-2.6.3-0ubuntu0.14.04.1)
OpenJDK 64-Bit Server VM (build 24.91-b01, mixed mode)
hduser#HDPMaster:~$ mvn --version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T08:41:47-08:00)
Maven home: /usr/local/apache-maven-339
Java version: 1.7.0_91, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.19.0-43-generic", arch: "amd64", family: "unix"
hduser#HDPMaster:~$ node --version
v0.10.25
Cloned Zeppelin from Git Hub
git clone https://github.com/apache/incubator-zeppelin.git
Initially, I had Maven 3.0 version. One of the threads suggested to have a version > 3.3..Installed the latest version (3.3.9).
executed the below command which is still failing
mvn clean package -DskipTests -Pspark-1.5.2 -Phadoop-2.7.1
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 32.205 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 36.457 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 21.499 s]
[INFO] Zeppelin: Spark dependencies ....................... SUCCESS [ 51.142 s]
[INFO] Zeppelin: Spark .................................... SUCCESS [01:01 min]
[INFO] Zeppelin: Markdown interpreter ..................... SUCCESS [ 3.089 s]
[INFO] Zeppelin: Angular interpreter ...................... SUCCESS [ 2.736 s]
[INFO] Zeppelin: Shell interpreter ........................ SUCCESS [ 2.813 s]
[INFO] Zeppelin: Hive interpreter ......................... SUCCESS [ 12.357 s]
[INFO] Zeppelin: Apache Phoenix Interpreter ............... SUCCESS [ 15.315 s]
[INFO] Zeppelin: PostgreSQL interpreter ................... SUCCESS [ 4.681 s]
[INFO] Zeppelin: Tajo interpreter ......................... SUCCESS [ 6.417 s]
[INFO] Zeppelin: Flink .................................... FAILURE [ 6.133 s]
[INFO] Zeppelin: Apache Ignite interpreter ................ SKIPPED
[INFO] Zeppelin: Kylin interpreter ........................ SKIPPED
[INFO] Zeppelin: Lens interpreter ......................... SKIPPED
[INFO] Zeppelin: Cassandra ................................ SKIPPED
[INFO] Zeppelin: Elasticsearch interpreter ................ SKIPPED
[INFO] Zeppelin: web Application .......................... SKIPPED
[INFO] Zeppelin: Server ................................... SKIPPED
[INFO] Zeppelin: Packaging distribution ................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 04:17 min
[INFO] Finished at: 2016-01-13T06:12:12-08:00
[INFO] Final Memory: 122M/341M
[INFO] ------------------------------------------------------------------------
[WARNING] The requested profile "spark-1.5.2" could not be activated because it does not exist.
[WARNING] The requested profile "hadoop-2.7.1" could not be activated because it does not exist.
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.1.4:compile (scala-compile-first) on project zeppelin-flink: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.1.4:compile failed: Plugin net.alchim31.maven:scala-maven-plugin:3.1.4 or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.apache.maven:maven-core:jar:3.0.4, org.apache.maven:maven-settings:jar:3.0.4, org.apache.maven:maven-settings-builder:jar:3.0.4, org.apache.maven:maven-repository-metadata:jar:3.0.4, org.apache.maven:maven-plugin-api:jar:3.0.4, org.apache.maven:maven-model-builder:jar:3.0.4, org.apache.maven:maven-aether-provider:jar:3.0.4, org.sonatype.aether:aether-spi:jar:1.13.1, org.sonatype.aether:aether-impl:jar:1.13.1, org.sonatype.aether:aether-api:jar:1.13.1, org.sonatype.sisu:sisu-inject-plexus:jar:2.3.0, org.apache.maven:maven-model:jar:3.0.4: Could not transfer artifact org.apache.maven:maven-core:jar:3.0.4 from/to central (https://repo.maven.apache.org/maven2): repo.maven.apache.org: Unknown host repo.maven.apache.org -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :zeppelin-flink
No idea where to go from here. Are there any changes required in any of the settings in zeppelin folder ? Suggestions please.
I don't know the reason behind it but this problem got solved. I just ran the below command in the morning and build was successful
mvn install -DskipTests
Not sure if there is a logic behind that but if someone knows please share that.
If you have cloned zeppelin from git, then you need to go to the 'master' folder(in command line) and then run the following command :
mvn clean package -DskipTests -Pspark-1.5.2 -Phadoop-2.7.1
Related
I'm following the documentation to build Flink 1.4 but instead Flink 1.5-SNAPSHOT is built.
What am I doing wrong?
[INFO] ------------------------------------------------------------------------
[INFO] Building flink-streaming-java 1.5-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) # flink-streaming-java_2.11 ---
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) # flink-streaming-java_2.11 ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) # flink-streaming-java_2.11 ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (dependency-convergence) # flink-streaming-java_2.11 ---
[INFO]
[INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) # flink-streaming-java_2.11 ---
[INFO] Highest basedir set to: /Users/amar/Downloads/flink-master
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) # flink-streaming-java_2.11 ---
[INFO]
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) # flink-streaming-java_2.11 ---
The documentation is not correct. It shows how to build the current master branch which is always the latest SNAPSHOT version (currently 1.5-SNAPSHOT). Before a new minor Flink version is released, we fork off a release branch. For Flink 1.4 this branch is called release-1.4.
You can build Flink 1.4-SNAPSHOT (the latest version in the dev branch with all bugfixes for Flink 1.4) by checking out the release-1.4 branch before calling Maven. So the commands should be:
git clone https://github.com/apache/flink
cd flink
git checkout release-1.4
mvn -DskipTests clean install
If you want to build a specific version, i.e., not the latest SNAPSHOT of a release, you can checkout a tag. For Flink 1.4.0 this would be git checkout release 1.4.0 instead of git checkout release 1.4.
I am trying to build Apache Zeppelin 0.8 SNAPSHOT using the source on GitHub.
Zeppelin Server build is failing with errors, any idea on the root cause for build failure?
[INFO] Zeppelin: Server ................................... FAILURE [
16.743 s]
[INFO] Zeppelin: Jupyter Support .......................... SKIPPED
[INFO] Zeppelin: Packaging distribution ................... SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 26.483 s
[INFO] Finished at: 2017-08-23T05:20:34-07:00
[INFO] Final Memory: 68M/341M
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile
(default-compile) on project zeppelin-server: Compilation failure:
Compilation failure: [ERROR]
/home/apache/zeppelin/zeppelin-server/src/main/java/org/apache/zeppelin/server/CorsFilter.java:[86,76]
cannot find symbol
[ERROR] symbol: method getStrictTransport()
[ERROR] location: variable zeppelinConfiguration of type
org.apache.zeppelin.conf.ZeppelinConfiguration
[ERROR]
/home/apache/zeppelin/zeppelin-server/src/main/java/org/apache/zeppelin/server/CorsFilter.java:[88,65]
cannot find symbol
[ERROR] symbol: method getXxssProtection() [ERROR] location:
variable zeppelinConfiguration of type
org.apache.zeppelin.conf.ZeppelinConfiguration
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with
the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with
the command
[ERROR] mvn -rf :zeppelin-server
Im trying to use jhipster for my backend project, one of the requirement is to use mssql. Based on my previous question :
https://github.com/jhipster/generator-jhipster/issues/4612#issuecomment-265076842
#ruddel did advise me to use this
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:sqlserver://localhost:1433;database=msqltest
username: SA
password: yourStrong(!)Password
hikari:
.........
jpa:
database-platform: org.hibernate.dialect.SQLServer2012Dialect
database: SQL_SERVER
.........
after setting this i need to run maven command so Jhipster initial entities can be generated for gateway project and Microservice by liquibase with this command as per guide by jhipster
liquibase:diff
and encountered this error:
[INFO] Settings
----------------------------
[INFO] driver: com.zaxxer.hikari.HikariConfig
[INFO] url: jdbc:sqlserver://localhost:1433;database=IASDB
[INFO] username: devuser
[INFO] password: *****
[INFO] use empty password: false
[INFO] properties file: null
[INFO] properties file will override? false
[INFO] prompt on non-local database? true
[INFO] clear checksums? false
[INFO] changeLogFile: src/main/resources/config/liquibase/master.xml
[INFO] context(s): null
[INFO] label(s): null
[INFO] referenceDriver: null
[INFO] referenceUrl: hibernate:spring:com.mbb.ias.domain?dialect=org.hibernate.dialect.SQLServer2012Dialect&hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.Spate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
[INFO] referenceUsername: null
[INFO] referencePassword: null
[INFO] referenceDefaultSchema: null
[INFO] diffChangeLogFile: src/main/resources/config/liquibase/changelog/20161208102850_changelog.xml
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.109 s
[INFO] Finished at: 2016-12-08T18:28:54+08:00
[INFO] Final Memory: 37M/342M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.liquibase:liquibase-maven-plugin:3.5.3:diff (default-cli) on project ias-gateway: Error setting up or running Liquibase: liquibase.exception.DatabaseExceptionnot find database driver: com.zaxxer.hikari.HikariConfig cannot be cast to java.sql.Driver -> [Help 1]
so i refer back to jhipster they did mention to change the setting inside pom.xml also which i did as per :
<groupId>org.liquibase</groupId>
<artifactId>liquibase-maven-plugin</artifactId>
<version>${liquibase.version}</version>
<configuration>
<changeLogFile>src/main/resources/config/liquibase/master.xml</changeLogFile>
<diffChangeLogFile>src/main/resources/config/liquibase/changelog/${maven.build.timestamp}_changelog.xml</diffChangeLogFile>
<driver>com.zaxxer.hikari.HikariConfig</driver>
<url>jdbc:sqlserver://localhost:1433;database=IASDB</url>
<defaultSchemaName>IASDB</defaultSchemaName>
<username>devuser</username>
<password>CIATopSecret</password>
<referenceUrl>hibernate:spring:com.mbb.ias.domain?dialect=org.hibernate.dialect.SQLServer2012Dialect&hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy&hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy</referenceUrl>
<verbose>true</verbose>
<logging>debug</logging>
</configuration>
it will hrow the error as per above msg or this :
[ERROR] Failed to execute goal org.liquibase:liquibase-maven-plugin:3.5.3:diff (default-cli) on project ias-gateway: Error setting up or running Liquibase: liquibase.exception.DatabaseExceptionnot find database driver: com.zaxxer.hikari.HikariConfig cannot be cast to java.sql.Driver -> [Help 1]
i did try to change my pom to use com.microsoft.SQLServerDriver, it throw me
[ERROR] Failed to execute goal org.liquibase:liquibase-maven-plugin:3.5.3:diff (default-cli) on project ias-gateway: Error setting up or running Liquibase: liquibase.exception.DatabaseExceptionnot find database driver: com.microsoft.SQLServerDriver -> [Help 1]
Anyone to advise how to change my pom? if i want to use sub generator, i need to generate the entities one by one? sorry for the noob question. here is my Jhipster info:
##### **JHipster configuration, a `.yo-rc.json` file generated in the root folder**
```yaml
{
"generator-jhipster": {
"jhipsterVersion": "3.12.0",
"baseName": "IASGateway",
"packageName": "com.mbb.ias",
"packageFolder": "com/mbb/ias",
"serverPort": "8080",
"authenticationType": "jwt",
"hibernateCache": "ehcache",
"clusteredHttpSession": false,
"websocket": false,
"databaseType": "sql",
"devDatabaseType": "mysql",
"prodDatabaseType": "mysql",
"searchEngine": false,
"messageBroker": "kafka",
"serviceDiscoveryType": "eureka",
"buildTool": "maven",
"enableSocialSignIn": false,
"jwtSecretKey": "3f2b1ccd09c528361c2b0f321f96fb62728e9830",
"useSass": false,
"applicationType": "gateway",
"testFrameworks": [
"gatling"
],
"jhiPrefix": "jhi",
"enableTranslation": false
}
}
```
##### **Entity configuration(s) `entityName.json` files generated in the `.jhipster` directory**
ls: no such file or directory: .jhipster/*.json
##### **Browsers and Operating System**
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
git version 2.10.2.windows.1
node: v7.2.1
npm: 3.10.10
bower: 1.8.0
gulp:
[18:56:33] CLI version 1.2.2
[18:56:33] Local version 3.9.1
yeoman: 1.8.5
For full MSSQL support, you should use v3.12.1 - your project was generated with v3.12.0, before we supported that database. You also shouldn't run liquibase:diff initially (it will fail until the app runs liquibase for the first time).
The following code is the pom configuration for MSSQL liquibase from a freshly generated project, the main difference I see is the driver tag:
<configuration>
<changeLogFile>src/main/resources/config/liquibase/master.xml</changeLogFile>
<diffChangeLogFile>src/main/resources/config/liquibase/changelog/${maven.build.timestamp}_changelog.xml</diffChangeLogFile>
<driver></driver>
<url>jdbc:sqlserver://localhost:1433;database=databaseName</url>
<defaultSchemaName></defaultSchemaName>
<username>SA</username>
<password>yourStrong(!)Password</password>
<referenceUrl>hibernate:spring:com.mycompany.myapp.domain?dialect=org.hibernate.dialect.SQLServerDialect&hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy&hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy</referenceUrl>
<verbose>true</verbose>
<logging>debug</logging>
</configuration>
I'm trying to deploy and run Google-pubsub sample code in Google app engine by this read me file. While running the deploy command I encounter below build error.
E:\java\cloud-pubsub-samples-java-master>mvn gcloud:deploy -Dgcloud.version=1 -DskipTests=true
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] pubsub-pull-sample
[INFO] Example for the Google Cloud Pub/Sub on Google App Engine.
[INFO] Cloud Dataflow Examples
[INFO] cloud-pubsub-samples-java
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/m2e/lifecycle-mapping/1.0.0/lifecycle-mapping-1.0.0.pom
[WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no dependency information available
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/m2e/lifecycle-mapping/1.0.0/lifecycle-mapping-1.0.0.jar
[WARNING] Failed to retrieve plugin descriptor for org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be resolved: Could not find artifact org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 in central (https://repo.maven.apache.org/maven2)
Downloading: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml
Downloading: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml
Downloaded: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-metadata.xml (13 KB at 9.8 KB/sec)
Downloaded: https://repo.maven.apache.org/maven2/org/codehaus/mojo/maven-metadata.xml (20 KB at 8.9 KB/sec)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] pubsub-pull-sample ................................. SKIPPED
[INFO] Example for the Google Cloud Pub/Sub on Google App Engine. SKIPPED
[INFO] Cloud Dataflow Examples ............................ SKIPPED
[INFO] cloud-pubsub-samples-java .......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.167 s
[INFO] Finished at: 2016-08-21T22:20:45+05:30
[INFO] Final Memory: 13M/245M
[INFO] ------------------------------------------------------------------------
[ERROR] No plugin found for prefix 'gcloud' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (C:\Users\JAISON.G\.m2\repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/NoPluginFoundForPrefixException
This means that you haven't included the gcloud maven plugin in pom.xml. But you should no longer use gcloud, rather using appengine. See the documentation here.
If I start my local google app engine dev server by using the App Engine maven plugin :
mvn appengine:devserver
and I try to launch a mapreduce job (using the appengine-mapreduce library), I get a java.security.AccessControlException: access denied ("javax.security.auth.AuthPermission" "getSubject") because the code try to invoke javax.security.auth.Subject.getSubject() method (see below for the full stack trace)
It's very weird because :
1) the javax.security.auth.Subject.getSubject() is in the white list of gae and can be invoked
2) If I start the server from intellij, I don't get this exception and the map reduce job is launched successfully
I tried with GAE 1.8.1, 1.8.2 and the latest 1.8.3 and respectively the same version for the maven google appengine plugin.
What is going wrong ?
The full stack trace :
[INFO] java.security.AccessControlException: access denied ("javax.security.auth.AuthPermission" "getSubject")
[INFO] at java.security.AccessControlContext.checkPermission(AccessControlContext.java:366)
[INFO] at java.security.AccessController.checkPermission(AccessController.java:560)
[INFO] at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
[INFO] at com.google.appengine.tools.development.DevAppServerFactory$CustomSecurityManager.checkPermission(DevAppServerFactory.java:383)
[INFO] at javax.security.auth.Subject.getSubject(Subject.java:287)
[INFO] at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:467)
[INFO] at org.apache.hadoop.mapreduce.JobContext.(JobContext.java:80)
[INFO] at com.google.appengine.tools.mapreduce.AppEngineJobContext.(AppEngineJobContext.java:123)
[INFO] at com.google.appengine.tools.mapreduce.AppEngineJobContext.createContextForNewJob(AppEngineJobContext.java:132)
[INFO] at com.google.appengine.tools.mapreduce.v2.impl.handlers.Controller.handleStart(Controller.java:85)
[INFO] at com.google.appengine.tools.mapreduce.v2.impl.handlers.Status.handleStartJob(Status.java:81)
[INFO] at com.google.appengine.tools.mapreduce.v2.impl.handlers.Status.handleCommand(Status.java:182)
[INFO] at com.google.appengine.tools.mapreduce.MapReduceServlet.doPost(MapReduceServlet.java:203)
[INFO] at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
[INFO] at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
[INFO] at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)