Currently, I'm setting up Zeppelin service. Zeppelin service daemon is able to start, but when I want to run my job from http://servername:9080/#/notebook/notebook name, then I have a runtime exception:
Exception in thread "main" java.lang.RuntimeException: core-site.xml not found
at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2869)
at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2838)
at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2715) .
Where should I define the path: #export HADOOP_CONF_DIR=/etc/hadoop/conf/, if anyone knows in specific.
Zeppelin version: 0.9
set HADOOP_CONF_DIR in zeppelin-env.sh
Related
I'm getting this error message when I'm trying to run my zeppelin:
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
WARN [2020-05-25 09:08:31,181] ({main} ZeppelinConfiguration.java[create]:159) - Failed to load configuration, proceeding with a default
INFO [2020-05-25 09:08:31,241] ({main} ZeppelinConfiguration.java[create]:171) - Server Host: 0.0.0.0
Exception in thread "main" java.lang.NumberFormatException: For input string: "tcp://172.30.239.172:80"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at org.apache.zeppelin.conf.ZeppelinConfiguration.getInt(ZeppelinConfiguration.java:248)
at org.apache.zeppelin.conf.ZeppelinConfiguration.getInt(ZeppelinConfiguration.java:243)
at org.apache.zeppelin.conf.ZeppelinConfiguration.getServerPort(ZeppelinConfiguration.java:327)
at org.apache.zeppelin.conf.ZeppelinConfiguration.create(ZeppelinConfiguration.java:173)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:129)
I'm deploying zeppelin without any configuration.
EDIT:
Additional error message after having set ZEPPELIN_PORT:
ERROR [2020-05-25 12:41:41,354] ({main} ZeppelinServer.java[main]:262) - Error while running jettyServer
java.net.SocketException: Permission denied
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:396)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:253)
It looks like you're using links with Docker compose, or something like, so it defines the ZEPPELIN_PORT environment variable for Zeppelin container that has a form of tcp://172.30.239.172:80. But this environment variable should contain only port, like, 80, or 8080.
To fix the problem try to rename linked container from zeppelin to something else, so it won't override the environment variable that is used by Zeppelin itself.
I am trying to run the simple code below after building everything from Flink's github master branch for various reasons. I get an exception below and I wonder what runs on port 9065? and How to fix this exception?
val dataStream = senv.fromElements(1, 2, 3, 4)
dataStream.countWindowAll(2).sum(0).print()
senv.execute("My streaming program")
Below is the Exception
Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit JobGraph.
at org.apache.flink.client.program.rest.RestClusterClient.lambda$submitJob$18(RestClusterClient.java:306)
at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at org.apache.flink.runtime.rest.RestClient.lambda$submitRequest$222(RestClient.java:196)
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:268)
at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:284)
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.CompletionException: java.net.ConnectException: Connection refused: localhost/127.0.0.1:9065
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniCompose(CompletableFuture.java:943)
at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:926)
... 16 more
Caused by: java.net.ConnectException: Connection refused: localhost/127.0.0.1:9065
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at org.apache.flink.shaded.netty4.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
I build it from the sources in the following way (just followed the instructions on Flink github page):
git clone https://github.com/apache/flink.git
cd flink
mvn clean package -DskipTests
cd build-target
./bin/start-scala-shell.sh local
Underlying distributed runtime is currently heavily worked on in master. Starting from 1.5 the default runtime will be the one known as FLIP6, therefore ocassionally some parts might not work. I think it would be very beneficial if you could create a JIRA ticket for this.
Just to add what runs on 9065 port, in the new architecture it is the default port of Dispatcher.
I had the same exception. My issue was that I had a port conflict when started the cluster with a docker image on my machine. So I had changed the port for rest in flink config file to use 8084 instead of 8081. When I did this the cluster would start up properly but I was unable to submit the job. When I killed the conflicting process and reverted the port back to 8081, I could submit jobs successfully
I got the same error.
Use jdk 1.8 for flink 1.7.2
I'm trying to run the solr 6.6.0 tutorial and after running:
bin/solr start -e cloud -noprompt
it starts solr on ports 8983 and 7574 but fails to create the getting started collection with the following error:
ERROR: Failed to create collection 'gettingstarted' due to: {10.1.20.105:7574_solr=org.apache.solr.client.solrj.SolrServerException:IOException occured when talking to server at: http://10.1.20.105:7574/solr}
ERROR: Failed to create collection using command: [-name, gettingstarted, -shards, 2, -replicationFactor, 2, -confname, gettingstarted, -confdir, data_driven_schema_configs, -configsetsDir, /Users/rcarey/solr-6.6.0/server/solr/configsets, -solrUrl, http://localhost:8983/solr]
It looks like its trying to create each replica on a different ip, rather than a different port on the same ip. 10.1.20.105 is not the IP that the 8983 replica is using. I'm not sure if theres something additional I need to configure for this so that it uses the one IP for both. I have the host set to localhost.
The Solr Admin is available on both http://localhost:8983/solr and http://localhost:7574/solr
I get the following in the log:
24/08/2017, 11:38:36 ERROR null OverseerCollectionMessageHandler Error from shard: http://10.1.20.105:7574/solr
24/08/2017, 11:38:36 ERROR null OverseerCollectionMessageHandler Error from shard: http://10.1.20.105:7574/solr
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://10.1.20.105:7574/solr
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:624)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to 10.1.20.105:7574 timed out
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:119)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 12 more
24/08/2017, 11:38:36 ERROR null OverseerCollectionMessageHandler Cleaning up collection [gettingstarted].
24/08/2017, 11:39:06 ERROR null CollectionsHandler Timed out waiting for new collection's replicas to become ACTIVE with timeout=30
Help me to fix this.
I had the same issue. In bin/solr.in.sh, I uncommented and set the following:
SOLR_HOST="localhost"
Then things worked because solr communicated with the server via "localhost" instead of an IP, which had a timeout. Fixing the error:
SolrServerException:IOException occured when talking to server at: http://YOUR_IP/solr
I am new to SOLR. I am building SOLR from source using solr-5.0.0-src.tgz. After running
ant compile
at solr-5.0.0/, I run
bin/solr start
at solr-5.0.0/solr/. And it says
Waiting to see Solr listening on port 8983 [/]
Started Solr server on port 8983 (pid=20151). Happy searching!
However, when visiting http://localhost:8983/solr/, I receive HTTP ERROR
HTTP ERROR: 503
Problem accessing /solr/. Reason:
Service Unavailable
Powered by Jetty://
And
bin/solr status
gives
Found 1 Solr nodes:
Solr process 20151 running on port 8983
Error: Could not find or load main class org.apache.solr.util.SolrCLI
I wonder if this is the reason admin page is unavailable? If so, how I could solve the problem. If not, what is it?
Thanks.
change to solr directory and run:
ant server
Then restart the server
bin/solr stop && bin/solr start
Check that everything is working:
bin/solr status
You have not mentioned the full stack trace...
Here it is ....
Exception in thread "main" java.lang.UnsupportedClassVersionError: org/apache/solr/util/SolrCLI : Unsupported major.minor version 51.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.solr.util.SolrCLI. Program will exit.
To fix the problem you need to upgrade the java ...to J2SE 7
We are using Apache CXF 2.5.2 for webservice client proxies. We use weblogic 10.3.4. To override the CXF logger we use the following option:
-Dorg.apache.cxf.Logger=org.apache.cxf.common.logging.Slf4jLogger
For the org.apache.cxf.common.logging.Slf4jLogger, we've included the cxf-common-utilities-2.5.2 in our build.
When we try to deploy to weblogic we get the following exception:
[ERROR] Target state: deploy failed on Server AdminServer
[ERROR] java.lang.NoClassDefFoundError: weblogic/wsee/jaxws/spi/WLSProvider
According to documentation, your WebService client should have wseeclient.jar as it's runtime dependency. If the problems is still there, then include wlfullclient.jar (see Creating a wlfullclient.jar for JDK 1.6 client applications ).