Failure to start Zeppelin - apache-zeppelin

When I start zeppelin o It starts on port 8080, it says port already in use. I tried changing port in zeppelin in config files, in zeppelin-site.xml.template made it too in zeppelin-env.sh.template adding the env line for the same port.
When I start zeppelin, I get OK, but if I see the open ports, It doesn't appear 8050 anywhere, so looks like It is still trying to deploy on port 8080
This is error I get when I click zeppelin.cmd
/zeppelin/interpreter/ignite/zeppelin-ignite_2.11-0.7.3.jar!/interpreter-settin
g.json
DEBUG [2018-05-06 07:33:49,111] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/jdbc/zeppelin-jdbc-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,126] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/kylin/zeppelin-kylin-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,281] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/lens/zeppelin-lens-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,431] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/livy/zeppelin-livy-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,466] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/md/zeppelin-markdown-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,754] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/pig/zeppelin-pig-0.7.3.jar!/interpreter-setting.json
WARN [2018-05-06 07:33:49,794] ({main} Interpreter.java[register]:406) - Static
initialization is deprecated for interpreter sql, You should change it to use i
nterpreter-setting.json in your jar or interpreter/{interpreter}/interpreter-set
ting.json
INFO [2018-05-06 07:33:49,794] ({main} InterpreterSettingManager.java[init]:305
) - Interpreter psql.sql found. class=org.apache.zeppelin.postgresql.PostgreSqlI
nterpreter
DEBUG [2018-05-06 07:33:52,889] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/python/zeppelin-python-0.7.3.jar!/interpreter-setting.jso
n
DEBUG [2018-05-06 07:33:53,234] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/sh/zeppelin-shell-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:55,279] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/spark/zeppelin-spark_2.11-0.7.3.jar!/interpreter-setting.
json
DEBUG [2018-05-06 07:33:55,284] ({main} InterpreterSettingManager.java[init]:320
) - Registered: psql.sql -> org.apache.zeppelin.postgresql.PostgreSqlInterpreter
. Properties: {postgresql.password={envName=null, propertyName=null, defaultValu
e=, description=The PostgreSQL user password}, postgresql.max.result={envName=nu
ll, propertyName=null, defaultValue=1000, description=Max number of SQL result t
o display.}, postgresql.user={envName=null, propertyName=null, defaultValue=gpad
min, description=The PostgreSQL user name}, postgresql.url={envName=null, proper
tyName=null, defaultValue=jdbc:postgresql://localhost:5432/, description=The URL
for PostgreSQL.}, postgresql.driver.name={envName=null, propertyName=null, defa
ultValue=org.postgresql.Driver, description= JDBC Driver Name}}
INFO [2018-05-06 07:33:55,289] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name ignite
INFO [2018-05-06 07:33:55,289] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name python
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name jdbc
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name psql
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name lens
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name pig
INFO [2018-05-06 07:33:55,299] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name flink
INFO [2018-05-06 07:33:55,299] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name angular
INFO [2018-05-06 07:33:55,299] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name livy
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name file
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name elasticsearch
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name cassandra
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name sh
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name spark
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name md
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name alluxio
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name bigquery
INFO [2018-05-06 07:33:55,314] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name hbase
INFO [2018-05-06 07:33:55,314] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name kylin
INFO [2018-05-06 07:33:55,450] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group md : id=2DEPH6PYC, name=md
INFO [2018-05-06 07:33:55,453] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group psql : id=2DD7ZJKPA, name=psql
INFO [2018-05-06 07:33:55,457] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group python : id=2DCAMH4GY, name=python
INFO [2018-05-06 07:33:55,463] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group file : id=2DDZH8WCB, name=file
INFO [2018-05-06 07:33:55,472] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group elasticsearch : id=2DE3BGT7H, name=elasticsearch
INFO [2018-05-06 07:33:55,476] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group bigquery : id=2DEVZXKAG, name=bigquery
INFO [2018-05-06 07:33:55,480] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group hbase : id=2DC4M79BE, name=hbase
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group sh : id=2DD1MA4VU, name=sh
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group pig : id=2DCNP1WD2, name=pig
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group alluxio : id=2DC7G4QW3, name=alluxio
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group livy : id=2DEQ54GYT, name=livy
INFO [2018-05-06 07:33:55,487] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group flink : id=2DDWBXQYG, name=flink
INFO [2018-05-06 07:33:55,487] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group spark : id=2DF55VV2T, name=spark
INFO [2018-05-06 07:33:55,487] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group ignite : id=2DCJEAGTV, name=ignite
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group kylin : id=2DC5T3GZV, name=kylin
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group angular : id=2DFHJM4RD, name=angular
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group lens : id=2DC9V3E9V, name=lens
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group jdbc : id=2DEECFK3N, name=jdbc
INFO [2018-05-06 07:33:55,497] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group cassandra : id=2DCH1RQVX, name=cassandra
INFO [2018-05-06 07:33:55,537] ({main} InterpreterFactory.java[<init>]:130) - s
hiroEnabled: false
INFO [2018-05-06 07:33:55,858] ({main} VfsLog.java[info]:138) - Using "C:\Users
\Weltinfo\AppData\Local\Temp\vfs_cache" as temporary files store.
INFO [2018-05-06 07:33:56,055] ({main} GitNotebookRepo.java[<init>]:63) - Openi
ng a git repo at '/zeppelin/notebook'
INFO [2018-05-06 07:33:56,578] ({main} NotebookAuthorization.java[loadFromFile]
:96) - C:\zeppelin\conf\notebook-authorization.json
INFO [2018-05-06 07:33:56,581] ({main} Credentials.java[loadFromFile]:102) - C:
\zeppelin\conf\credentials.json
INFO [2018-05-06 07:33:56,710] ({main} StdSchedulerFactory.java[instantiate]:11
84) - Using default implementation for ThreadExecutor
INFO [2018-05-06 07:33:56,720] ({main} SimpleThreadPool.java[initialize]:268) -
Job execution threads will use class loader of thread: main
INFO [2018-05-06 07:33:56,775] ({main} SchedulerSignalerImpl.java[<init>]:61) -
Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignale
rImpl
INFO [2018-05-06 07:33:56,798] ({main} QuartzScheduler.java[<init>]:240) - Quar
tz Scheduler v.2.2.1 created.
INFO [2018-05-06 07:33:56,802] ({main} RAMJobStore.java[initialize]:155) - RAMJ
obStore initialized.
INFO [2018-05-06 07:33:56,806] ({main} QuartzScheduler.java[initialize]:305) -
Scheduler meta-data: Quartz Scheduler (v2.2.1) 'DefaultQuartzScheduler' with ins
tanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persis
tence. and is not clustered.
INFO [2018-05-06 07:33:56,810] ({main} StdSchedulerFactory.java[instantiate]:13
39) - Quartz scheduler 'DefaultQuartzScheduler' initialized from default resourc
e file in Quartz package: 'quartz.properties'
INFO [2018-05-06 07:33:56,813] ({main} StdSchedulerFactory.java[instantiate]:13
43) - Quartz scheduler version: 2.2.1
INFO [2018-05-06 07:33:56,815] ({main} QuartzScheduler.java[start]:575) - Sched
uler DefaultQuartzScheduler_$_NON_CLUSTERED started.
INFO [2018-05-06 07:33:57,388] ({main} FolderView.java[createFolder]:107) - Cre
ate folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,394] ({main} FolderView.java[createFolder]:107) - Cre
ate folder /
INFO [2018-05-06 07:33:57,396] ({main} Folder.java[setParent]:168) - Set parent
of / to /
INFO [2018-05-06 07:33:57,397] ({main} Folder.java[setParent]:168) - Set parent
of Zeppelin Tutorial to /
INFO [2018-05-06 07:33:57,398] ({main} Folder.java[addNote]:184) - Add note 2A9
4M5J1Z to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,456] ({main} Folder.java[addNote]:184) - Add note 2BW
JFTXKJ to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,504] ({main} Folder.java[addNote]:184) - Add note 2BY
EZ5EVK to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,528] ({main} Folder.java[addNote]:184) - Add note 2C2
AUG798 to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,557] ({main} Folder.java[addNote]:184) - Add note 2C3
5YU814 to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,574] ({main} Folder.java[addNote]:184) - Add note 2C5
7UKYWR to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,575] ({main} Notebook.java[<init>]:127) - Notebook in
dexing started...
INFO [2018-05-06 07:33:58,305] ({main} LuceneSearch.java[addIndexDocs]:305) - I
ndexing 6 notebooks took 729ms
INFO [2018-05-06 07:33:58,305] ({main} Notebook.java[<init>]:129) - Notebook in
dexing finished: 6 indexed in 0s
INFO [2018-05-06 07:33:58,603] ({main} ServerImpl.java[initDestination]:94) - S
etting the server's publish address to be /
mai 06, 2018 7:33:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFOS: Scanning for root resource and provider classes in the packages:
org.apache.zeppelin.rest
mai 06, 2018 7:33:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasse
s
INFOS: Root resource classes found:
class org.apache.zeppelin.rest.SecurityRestApi
class org.apache.zeppelin.rest.HeliumRestApi
class org.apache.zeppelin.rest.ConfigurationsRestApi
class org.apache.zeppelin.rest.LoginRestApi
class org.apache.zeppelin.rest.InterpreterRestApi
class org.apache.zeppelin.rest.NotebookRepoRestApi
class org.apache.zeppelin.rest.CredentialRestApi
class org.apache.zeppelin.rest.NotebookRestApi
class org.apache.zeppelin.rest.ZeppelinRestApi
mai 06, 2018 7:33:58 AM com.sun.jersey.api.core.ScanningResourceConfig init
INFOS: No provider classes found.
mai 06, 2018 7:33:59 AM com.sun.jersey.server.impl.application.WebApplicationImp
l _initiate
INFOS: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012 05:14 PM'
mai 06, 2018 7:34:00 AM com.sun.jersey.spi.inject.Errors processErrorMessages
AVERTISSEMENT: The following warnings have been detected with resource and/or pr
ovider classes:
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.zeppel
in.rest.CredentialRestApi.getCredentials(java.lang.String) throws java.io.IOExce
ption,java.lang.IllegalArgumentException, should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.zeppel
in.rest.InterpreterRestApi.listInterpreter(java.lang.String), should not consume
any entity.
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.ze
ppelin.rest.NotebookRestApi.createNote(java.lang.String) throws java.io.IOExcept
ion, with URI template, "/", is treated as a resource method
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.ze
ppelin.rest.NotebookRestApi.getNoteList() throws java.io.IOException, with URI t
emplate, "/", is treated as a resource method
INFO [2018-05-06 07:34:00,216] ({main} ContextHandler.java[doStart]:744) - Star
ted o.e.j.w.WebAppContext#140e5a13{/,file:/C:/zeppelin/webapps/webapp/,AVAILABLE
}{C:\zeppelin\zeppelin-web-0.7.3.war}
WARN [2018-05-06 07:34:00,227] ({main} AbstractLifeCycle.java[setFailed]:212) -
FAILED ServerConnector#1317b708{HTTP/1.1}{0.0.0.0:8080}: java.net.BindException
: Address already in use: bind
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:32
1)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNet
workConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java
:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:18
9)
WARN [2018-05-06 07:34:00,233] ({main} AbstractLifeCycle.java[setFailed]:212) -
FAILED org.eclipse.jetty.server.Server#44a6a68e: java.net.BindException: Addres
s already in use: bind
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:32
1)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNet
workConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java
:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:18
9)
ERROR [2018-05-06 07:34:00,236] ({main} ZeppelinServer.java[main]:194) - Error w
hile running jettyServer
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:32
1)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNet
workConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java
:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:18
9)

"zeppelin-site.xml.template" and "zeppelin-env.sh.template" are not the files that are actual used when running Zeppelin. They are - as their name suggest - templates that should help to build the actual configuration files. They should not be changed.
Instead, create a copy of these files that are named "zeppelin-site.xml" or "zeppelin-env.sh" respectively and adopt these files according to your needs. During the next restart of Zeppelin the new settings should be picked up.

Related

APACHE zeppelin JSP support not found

I have a error relative to a JSP
support not found
log:
INFO [2016-03-11 11:01:26,017] ({main} StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
INFO [2016-03-11 11:01:26,018] ({main} StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version: 2.2.1
INFO [2016-03-11 11:01:26,019] ({main} QuartzScheduler.java[start]:575) - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
INFO [2016-03-11 11:01:26,173] ({main} Notebook.java[]:107) - Notebook indexing started...
INFO [2016-03-11 11:01:26,299] ({main} LuceneSearch.java[addIndexDocs]:285) - Indexing 0 notebooks took 125ms
INFO [2016-03-11 11:01:26,313] ({main} Notebook.java[]:109) - Notebook indexing finished: 0 indexed in 0s
INFO [2016-03-11 11:01:27,225] ({main} ServerImpl.java[initDestination]:94) - Setting the server's publish address to be /
INFO [2016-03-11 11:01:28,112] ({main} StandardDescriptorProcessor.java[visitServlet]:284) - NO JSP Support for /srv/zeppelin/tmp/webapp, did not find org.apache.jasper.servlet.JspServlet
INFO [2016-03-11 11:01:31,655] ({main} AbstractConnector.java[doStart]:338) - Started SelectChannelConnector#192.168.1.105:8090
INFO [2016-03-11 11:01:31,677] ({main} ZeppelinServer.java[main]:115) - Done, zeppelin server started
zeppelin-site.xml
<configuration>
<property>
<name>zeppelin.server.addr</name>
<value>192.168.1.105</value>
<description>Server address</description>
</property>
<property>
<name>zeppelin.server.port</name>
<value>8090</value>
<description>Server port.</description>
</property>
<property>
<name>zeppelin.server.context.path</name>
<value>/srv/zeppelin/tmp/webapp</value>
<description>Context Path of the Web Application</description>
</property>
<property>
<name>zeppelin.war.tempdir</name>
<value>/srv/zeppelin/tmp</value>
<description>Location of jetty temporary directory</description>
</property>
<property>
<name>zeppelin.notebook.dir</name>
<value>/srv/zeppelin/notebook</value>
<description>path or URI for notebook persist</description>
</property>
<property>
<name>zeppelin.notebook.homescreen</name>
<value></value>
<description>id of notebook to be displayed in homescreen. ex) 2A94M5J1Z Empty value displays default home screen</description>
</property>
<property>
<name>zeppelin.notebook.homescreen.hide</name>
<value>false</value>
<description>hide homescreen notebook from list when this value set to true</description>
</property>
I solve the problem....
<property>
<name>zeppelin.server.context.path</name>
<value>/</value>
<description>Context Path of the Web Application</description>
</property>
i think you have to put the root path and somewhere in the filesystem the software find the jsp support

Zeppelin disconnected

I downloaded zeppelin binary package and started it. However it's disconnected.
Here is the log:
INFO [2016-01-12 14:37:56,592] ({main} QuartzScheduler.java[initialize]:305) - Scheduler meta-data: Quartz Scheduler (v2.2.1) 'Defau
ltQuartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.
INFO [2016-01-12 14:37:56,593] ({main} StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler 'DefaultQuartzScheduler' initi
alized from default resource file in Quartz package: 'quartz.properties'
INFO [2016-01-12 14:37:56,593] ({main} StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version: 2.2.1
INFO [2016-01-12 14:37:56,593] ({main} QuartzScheduler.java[start]:575) - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
INFO [2016-01-12 14:37:56,805] ({main} ServerImpl.java[initDestination]:94) - Setting the server's publish address to be /
INFO [2016-01-12 14:37:56,888] ({main} WebInfConfiguration.java[unpack]:478) - Extract jar:file:/data/users/huser/zeppelin/zeppelin-
web-0.5.5-incubating.war!/ to /tmp/jetty-0.0.0.0-8080-zeppelin-web-0.5.5-incubating.war-_-any-/webapp
INFO [2016-01-12 14:37:57,040] ({main} StandardDescriptorProcessor.java[visitServlet]:284) - NO JSP Support for /, did not find org.
apache.jasper.servlet.JspServlet
INFO [2016-01-12 14:37:57,941] ({main} AbstractConnector.java[doStart]:338) - Started SelectChannelConnector#0.0.0.0:8080
INFO [2016-01-12 14:37:57,941] ({main} ZeppelinServer.java[main]:108) - Started zeppelin server
I searched and found that port+1 is the websocket port. However, it's not listening:
netstat -na | grep 8080
tcp 0 0 :::8080 :::* LISTEN
netstat -na | grep 8081
noting...
And the web return the error:
WebSocket connection to 'ws://ip:8080/ws' failed: Establishing a tunnel via proxy server failed.
Can anyone help?
Thanks.
Make sure port forwarding command executes on yr client system all the time you use zeppelin
ssh -i ~/keys/mykey.pem -N -L 8122:xx.xxx.xxx.xxx:8890 hadoop#yy.yyy.yyy.yyy

SolrCloud - How to resolve "Could not find collection configName"

I am trying to start Solr in SolrCloud mode. I have created a new collection from collection1 and changed its name in file core.properties by setting the property name=logmail.
But when I start Solr, I am getting the following error
$ java -Dcollection.configName=logmail -DzkRun -Dnumshards=2 -DBootstrap_confdir=./solr/logmail/conf -jar start.jar
2165 [main] INFO org.apache.solr.common.cloud.ZkStateReader –
Updating cluster state from ZooKeeper... 2179
[OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001]
INFO org.apache.solr.cloud.Overseer – Starting to work on the main
queue 2197 [main] INFO org.apache.solr.core.CoresLocator – Looking
for core definitions underneath /home/rahul/Desktop/dev/solrcloud/solr
2203 [main] INFO org.apache.solr.core.CoresLocator – Found core
logmail in /home/rahul/Desktop/dev/solrcloud/solr/logmail/ 2204 [main]
INFO org.apache.solr.core.CoresLocator – Found core collection1 in
/home/rahul/Desktop/dev/solrcloud/solr/collection1/ 2204 [main] INFO
org.apache.solr.core.CoresLocator – Found 2 core definitions 2207
[coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController
– publishing core=logmail state=down collection=logmail 2207
[coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController
– publishing core=collection1 state=down collection=collection1 2208
[coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController
– numShards not found on descriptor - reading it from system property
2208 [coreLoadExecutor-6-thread-2] INFO
org.apache.solr.cloud.ZkController – numShards not found on
descriptor - reading it from system property 2214
[coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController
– look for our core node name 2214 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – waiting to find shard id in
clusterstate for logmail 2214 [zkCallback-2-thread-1] INFO
org.apache.solr.cloud.DistributedQueue – NodeChildrenChanged fired on
path /overseer/queue state SyncConnected 2215
[coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController
– Check for collection zkNode:logmail 2222
[coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController
– look for our core node name 2222 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – Creating collection in
ZooKeeper:logmail 2222 [coreLoadExecutor-6-thread-2] INFO
org.apache.solr.cloud.ZkController – waiting to find shard id in
clusterstate for collection1 2223 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – Looking for collection
configName 2223 [coreLoadExecutor-6-thread-2] INFO
org.apache.solr.cloud.ZkController – Check for collection
zkNode:collection1 2224 [coreLoadExecutor-6-thread-2] INFO
org.apache.solr.cloud.ZkController – Creating collection in
ZooKeeper:collection1 2224 [coreLoadExecutor-6-thread-2] INFO
org.apache.solr.cloud.ZkController – Looking for collection
configName 2225 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – Could not find collection
configName - pausing for 3 seconds and trying again - try: 1 2226
[coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController
– Could not find collection configName - pausing for 3 seconds and
trying again - try: 1 2226
[OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001]
INFO org.apache.solr.cloud.Overseer – Update state numShards=null
message={ "core":"logmail", "roles":null,
"base_url":"http://127.0.1.1:8983/solr",
"node_name":"127.0.1.1:8983_solr", "state":"down", "shard":null,
"collection":"logmail", "operation":"state"} 2226
[OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001]
INFO org.apache.solr.cloud.Overseer – node=core_node1 is already
registered 2227
[OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001]
INFO org.apache.solr.cloud.Overseer – shard=shard1 is already
registered 2255 [zkCallback-2-thread-1] INFO
org.apache.solr.common.cloud.ZkStateReader – A cluster state change:
WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size:
1) 2268
[OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001]
INFO org.apache.solr.cloud.Overseer – Update state numShards=null
message={ "core":"collection1", "roles":null,
"base_url":"http://127.0.1.1:8983/solr",
"node_name":"127.0.1.1:8983_solr", "state":"down", "shard":null,
"collection":"collection1", "operation":"state"} 2268
[OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001]
INFO org.apache.solr.cloud.Overseer – node=core_node1 is already
registered 2269
[OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001]
INFO org.apache.solr.cloud.Overseer – shard=shard1 is already
registered 2288 [zkCallback-2-thread-1] INFO
org.apache.solr.cloud.DistributedQueue – NodeChildrenChanged fired on
path /overseer/queue state SyncConnected 2318 [zkCallback-2-thread-1]
INFO org.apache.solr.common.cloud.ZkStateReader – A cluster state
change: WatchedEvent state:SyncConnected type:NodeDataChanged
path:/clusterstate.json, has occurred - updating... (live nodes size:
1) 5227 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – Could not find collection
configName - pausing for 3 seconds and trying again - try: 2 5228
[coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController
– Could not find collection configName - pausing for 3 seconds and
trying again - try: 2 8229 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – Could not find collection
configName - pausing for 3 seconds and trying again - try: 3 8229
[coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController
– Could not find collection configName - pausing for 3 seconds and
trying again - try: 3 11232 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – Could not find collection
configName - pausing for 3 seconds and trying again - try: 4 11232
[coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController
– Could not find collection configName - pausing for 3 seconds and
trying again - try: 4 14237 [coreLoadExecutor-6-thread-1] INFO
org.apache.solr.cloud.ZkController – Could not find collection
configName - pausing for 3 seconds and trying again - try: 5 14237
[coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController
– Could not find collection configName - pausing for 3 seconds and
trying again - try: 5 17237 [coreLoadExecutor-6-thread-1] ERROR
org.apache.solr.cloud.ZkController – Could not find configName for
collection logmail 17238 [coreLoadExecutor-6-thread-2] ERROR
org.apache.solr.cloud.ZkController – Could not find configName for
collection collection1 17240 [coreLoadExecutor-6-thread-1] ERROR
org.apache.solr.core.CoreContainer – Error creating core [logmail]:
Could not find configName for collection logmail found:null
org.apache.solr.common.cloud.ZooKeeperException: Could not find
configName for collection logmail found:null
It looks like there may be a discrepancy between what Solr has on the filesystem for collections based on your commands and what is in zookeeper.
These are hard to fix; if possible I would recommend to delete your configuration files out of zookeeper and reload them.
You have a typo in your command. This should do the trick:
$ java -Dcollection.configName=logmail -DzkRun -Dnumshards=2 -Dbootstrap_confdir=./solr/logmail/conf -jar start.jar

Loading solr configs in Cloudera SolrCloud

We try to import our data into SolrCloud using MapReduce batch indexing. We face a problem at the reduce phase, that solr.xml cannot be found. We create a 'twitter' collection but looking at the logs, after it failed to load in solr.xml, it uses the default one and tries to create 'collection1' (failed) and 'core1' (success) SolrCore. I'm not sure if we need to create our own solr.xml and where to put it (we try to put it at several places but it seems not to load in). Below is the log:
2022 [main] INFO org.apache.solr.hadoop.HeartBeater - Heart beat reporting class is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl
2025 [main] INFO org.apache.solr.hadoop.SolrRecordWriter - Using this unpacked directory as solr home: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip
2025 [main] INFO org.apache.solr.hadoop.SolrRecordWriter - Creating embedded Solr server with solrHomeDir: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip, fs: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_-1828461666_1, ugi=nguyen (auth:SIMPLE)]], outputShardDir: hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014
2029 [Thread-64] INFO org.apache.solr.hadoop.HeartBeater - HeartBeat thread running
2030 [Thread-64] INFO org.apache.solr.hadoop.HeartBeater - Issuing heart beat for 1 threads
2083 [main] INFO org.apache.solr.core.SolrResourceLoader - new SolrResourceLoader for directory: '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/'
2259 [main] INFO org.apache.solr.hadoop.SolrRecordWriter - Constructed instance information solr.home /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip (/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip), instance dir /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/, conf dir /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/conf/, writing index to solr.data.dir hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data, with permdir hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014
2266 [main] INFO org.apache.solr.core.ConfigSolr - Loading container configuration from /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/solr.xml
2267 [main] INFO org.apache.solr.core.ConfigSolr - /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/solr.xml does not exist, using default configuration
2505 [main] INFO org.apache.solr.core.CoreContainer - New CoreContainer 696103669
2505 [main] INFO org.apache.solr.core.CoreContainer - Loading cores into CoreContainer [instanceDir=/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/]
2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting socketTimeout to: 0
2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting urlScheme to: http://
2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting connTimeout to: 0
2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting maxConnectionsPerHost to: 20
2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting corePoolSize to: 0
2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting maximumPoolSize to: 2147483647
2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting maxThreadIdleTime to: 5
2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting sizeOfQueue to: -1
2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting fairnessPolicy to: false
2527 [main] INFO org.apache.solr.client.solrj.impl.HttpClientUtil - Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
2648 [main] INFO org.apache.solr.logging.LogWatcher - Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
2676 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.CoreContainer - Creating SolrCore 'collection1' using instanceDir: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1
2677 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.SolrResourceLoader - new SolrResourceLoader for directory: '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/'
2691 [coreLoadExecutor-3-thread-1] ERROR org.apache.solr.core.CoreContainer - Failed to load file /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/solrconfig.xml
2693 [coreLoadExecutor-3-thread-1] ERROR org.apache.solr.core.CoreContainer - Unable to create core: collection1
org.apache.solr.common.SolrException: Could not load config for solrconfig.xml
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:596)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:661)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:368)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:360)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/conf/', cwd=/data/05/mapred/local/taskTracker/nguyen/jobcache/job_201311191613_0320/attempt_201311191613_0320_r_000014_0/work
at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:322)
at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:287)
at org.apache.solr.core.Config.<init>(Config.java:116)
at org.apache.solr.core.Config.<init>(Config.java:86)
at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:120)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:593)
... 11 more
2695 [coreLoadExecutor-3-thread-1] ERROR org.apache.solr.core.CoreContainer - null:org.apache.solr.common.SolrException: Unable to create core: collection1
at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1158)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:670)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:368)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:360)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.apache.solr.common.SolrException: Could not load config for solrconfig.xml
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:596)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:661)
... 10 more
Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/conf/', cwd=/data/05/mapred/local/taskTracker/nguyen/jobcache/job_201311191613_0320/attempt_201311191613_0320_r_000014_0/work
at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:322)
at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:287)
at org.apache.solr.core.Config.<init>(Config.java:116)
at org.apache.solr.core.Config.<init>(Config.java:86)
at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:120)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:593)
... 11 more
2697 [main] INFO org.apache.solr.core.CoreContainer - Creating SolrCore 'core1' using instanceDir: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip
2697 [main] INFO org.apache.solr.core.SolrResourceLoader - new SolrResourceLoader for directory: '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/'
2751 [main] INFO org.apache.solr.core.SolrConfig - Adding specified lib dirs to ClassLoader
2752 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/extraction/lib).
2752 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist).
2752 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/clustering/lib).
2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist).
2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/langid/lib).
2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist).
2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/velocity/lib).
2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist).
2785 [main] INFO org.apache.solr.update.SolrIndexConfig - IndexWriter infoStream solr logging is enabled
2790 [main] INFO org.apache.solr.core.SolrConfig - Using Lucene MatchVersion: LUCENE_44
2869 [main] INFO org.apache.solr.core.Config - Loaded SolrConfig: solrconfig.xml
2879 [main] INFO org.apache.solr.schema.IndexSchema - Reading Solr Schema from schema.xml
2937 [main] INFO org.apache.solr.schema.IndexSchema - [core1] Schema name=twitter
3352 [main] INFO org.apache.solr.schema.IndexSchema - unique key field: id
3471 [main] INFO org.apache.solr.schema.FileExchangeRateProvider - Reloading exchange rates from file currency.xml
3478 [main] INFO org.apache.solr.schema.FileExchangeRateProvider - Reloading exchange rates from file currency.xml
3635 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - Solr Kerberos Authentication disabled
3636 [main] INFO org.apache.solr.core.JmxMonitoredMap - No JMX servers found, not exposing Solr information with JMX.
3652 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - creating directory factory for path hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data
3686 [main] INFO org.apache.solr.core.CachingDirectoryFactory - return new directory for hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data
3711 [main] WARN org.apache.solr.core.SolrCore - [core1] Solr index directory 'hdfs:/master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data/index' doesn't exist. Creating new index...
3719 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - creating directory factory for path hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data/index
3719 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - Number of slabs of block cache [1] with direct memory allocation set to [true]
3720 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - Block cache target memory usage, slab size of [134217728] will allocate [1] slabs and use ~[134217728] bytes
3721 [main] INFO org.apache.solr.store.blockcache.BufferStore - Initializing the 1024 buffers with [8192] buffers.
3740 [main] INFO org.apache.solr.store.blockcache.BufferStore - Initializing the 8192 buffers with [8192] buffers.
3891 [main] INFO org.apache.solr.core.CachingDirectoryFactory - return new directory for hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data/index
3988 [main] INFO org.apache.solr.update.LoggingInfoStream - [IFD][main]: init: current segments file is "null"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper#65b01d5d
3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IFD][main]: now checkpoint "" [0 segments ; isCommit = false]
3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IFD][main]: 0 msec to checkpoint
3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IW][main]: init: create=true
3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IW][main]:
dir=NRTCachingDirectory(org.apache.solr.store.hdfs.HdfsDirectory#17e5a6d8 lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory#7f117668; maxCacheMB=192.0 maxMergeSizeMB=16.0)
solr looks for solr.home parameter and searchs solrConfig.xml file there. if there is none it tries to load default configuration.
it looks like your solr home is
/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/
check that folder for solrconfig.xml file
if there is none, copy one from example directory of solr
if there is one, match the file/folder permissions with the server instance

zookeeper does not run?

I wanted to run a solr cloud with solr 4.3.0.
(I am using aws ubuntu-12.04-lts micro instances)
So I followed this toturial:
which basically says, start the zookeeper and connect the solr instances to it.
Here's how I start the zookeeper.
First I copied the config like described in the tutorial
sudo cp zookeeper-3.4.5/conf/zoo_sample.cfg zookeeper-3.4.5/conf/zoo.cfg
Then I started the zookeeper
ubuntu#ip-10-48-159-36:/opt$ sudo zookeeper-3.4.5/bin/zkServer.sh start
JMX enabled by default
Using config: /opt/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Looks fine so far.
I checked the status:
ubuntu#ip-10-48-159-36:/opt$ sudo zookeeper-3.4.5/bin/zkServer.sh status
JMX enabled by default
Using config: /opt/zookeeper-3.4.5/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
Which seems a bit weird already.
If I try to connect with the client (remote as well as local), its seems to work
ubuntu#ip-10-234-223-69:/opt$ zookeeper-3.4.5/bin/zkCli.sh -server ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181
Connecting to ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181
2013-06-07 11:07:01,996 [myid:] - INFO [main:Environment#100] - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2013-06-07 11:07:02,000 [myid:] - INFO [main:Environment#100] - Client environment:host.name=ip-10-234-223-69.eu-west-1.compute.internal
2013-06-07 11:07:02,000 [myid:] - INFO [main:Environment#100] - Client environment:java.version=1.6.0_27
2013-06-07 11:07:02,002 [myid:] - INFO [main:Environment#100] - Client environment:java.vendor=Sun Microsystems Inc.
2013-06-07 11:07:02,003 [myid:] - INFO [main:Environment#100] - Client environment:java.home=/usr/lib/jvm/java-6-openjdk-amd64/jre
2013-06-07 11:07:02,003 [myid:] - INFO [main:Environment#100] - Client environment:java.class.path=/opt/zookeeper-3.4.5/bin/../build/classes:/opt/zookeeper-3.4.5/bin/../build/lib/*.jar:/opt/zookeeper-3.4.5/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper-3.4.5/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper-3.4.5/bin/../lib/netty-3.2.2.Final.jar:/opt/zookeeper-3.4.5/bin/../lib/log4j-1.2.15.jar:/opt/zookeeper-3.4.5/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.5/bin/../zookeeper-3.4.5.jar:/opt/zookeeper-3.4.5/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.5/bin/../conf:
2013-06-07 11:07:02,004 [myid:] - INFO [main:Environment#100] - Client environment:java.library.path=/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2013-06-07 11:07:02,008 [myid:] - INFO [main:Environment#100] - Client environment:java.io.tmpdir=/tmp
2013-06-07 11:07:02,009 [myid:] - INFO [main:Environment#100] - Client environment:java.compiler=<NA>
2013-06-07 11:07:02,018 [myid:] - INFO [main:Environment#100] - Client environment:os.name=Linux
2013-06-07 11:07:02,019 [myid:] - INFO [main:Environment#100] - Client environment:os.arch=amd64
2013-06-07 11:07:02,019 [myid:] - INFO [main:Environment#100] - Client environment:os.version=3.2.0-40-virtual
2013-06-07 11:07:02,020 [myid:] - INFO [main:Environment#100] - Client environment:user.name=ubuntu
2013-06-07 11:07:02,020 [myid:] - INFO [main:Environment#100] - Client environment:user.home=/home/ubuntu
2013-06-07 11:07:02,021 [myid:] - INFO [main:Environment#100] - Client environment:user.dir=/opt
2013-06-07 11:07:02,029 [myid:] - INFO [main:ZooKeeper#438] - Initiating client connection, connectString=ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher#182d9c06
Welcome to ZooKeeper!
2013-06-07 11:07:02,074 [myid:] - INFO [main-SendThread(ip-10-48-159-36.eu-west-1.compute.internal:2181):ClientCnxn$SendThread#966] - Opening socket connection to server ip-10-48-159-36.eu-west-1.compute.internal/10.48.159.36:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
[zk: ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181(CONNECTING) 0] 2013-06-07 11:07:32,100 [myid:] - INFO [main-SendThread(ip-10-48-159-36.eu-west-1.compute.internal:2181):ClientCnxn$SendThread#1083] - Client session timed out, have not heard from server in 30038ms for sessionid 0x0, closing socket connection and attempting reconnect
2013-06-07 11:07:33,204 [myid:] - INFO [main-SendThread(ip-10-48-159-36.eu-west-1.compute.internal:2181):ClientCnxn$SendThread#966] - Opening socket connection to server ip-10-48-159-36.eu-west-1.compute.internal/10.48.159.36:2181. Will not attempt to authenticate using SASL (unknown error)
Now I tried to connect a solr instance to it. In the web interface of tomcat7 it only tells me "503 - Server is shutting down", so I checked the solr logs
2013-06-07 11:16:36,065 [pool-2-thread-1] INFO org.apache.solr.servlet.SolrDispatchFilter . SolrDispatchFilter.init()
2013-06-07 11:16:36,100 [pool-2-thread-1] INFO org.apache.solr.core.SolrResourceLoader . Using JNDI solr.home: /opt/solr-4.3.0/example/solr
2013-06-07 11:16:36,132 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . looking for solr config file: /opt/solr-4.3.0/example/solr/solr.xml
2013-06-07 11:16:36,138 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . New CoreContainer 1285984216
2013-06-07 11:16:36,146 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . Loading CoreContainer using Solr Home: '/opt/solr-4.3.0/example/solr/'
2013-06-07 11:16:36,152 [pool-2-thread-1] INFO org.apache.solr.core.SolrResourceLoader . new SolrResourceLoader for directory: '/opt/solr-4.3.0/example/solr/'
2013-06-07 11:16:36,567 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting socketTimeout to: 0
2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting urlScheme to: http://
2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting connTimeout to: 0
2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting maxConnectionsPerHost to: 20
2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting corePoolSize to: 0
2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting maximumPoolSize to: 2147483647
2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting maxThreadIdleTime to: 5
2013-06-07 11:16:36,569 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting sizeOfQueue to: -1
2013-06-07 11:16:36,569 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting fairnessPolicy to: false
2013-06-07 11:16:36,578 [pool-2-thread-1] INFO org.apache.solr.client.solrj.impl.HttpClientUtil . Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
2013-06-07 11:16:36,879 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . Registering Log Listener
2013-06-07 11:16:36,881 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . Zookeeper client=ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181
2013-06-07 11:16:36,888 [pool-2-thread-1] INFO org.apache.solr.client.solrj.impl.HttpClientUtil . Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=0&connTimeout=0
2013-06-07 11:16:37,040 [pool-2-thread-1] INFO org.apache.solr.common.cloud.ConnectionManager . Waiting for client to connect to ZooKeeper
2013-06-07 11:16:52,046 [pool-2-thread-1] ERROR org.apache.solr.servlet.SolrDispatchFilter . Could not start Solr. Check solr/home property and the logs
2013-06-07 11:16:52,103 [pool-2-thread-1] ERROR org.apache.solr.core.SolrCore . null:java.lang.RuntimeException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 within 15000 ms
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:130)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:88)
at org.apache.solr.cloud.ZkController.<init>(ZkController.java:170)
at org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:242)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:358)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:326)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:124)
at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258)
at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382)
at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103)
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:679)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 within 15000 ms
at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:173)
at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:127)
... 25 more
2013-06-07 11:16:52,104 [pool-2-thread-1] INFO org.apache.solr.servlet.SolrDispatchFilter . SolrDispatchFilter.init() done
What does it tell me?
On the same instance I just connected with the client successfully... :(
So where is the problem?
[Edit:]
Instead of using amazons ec**.amazon.* address I used the network addresses 10.X.X.X for telling solr where the zookeeper is.
It seems to work.
You have your answer - Your ZooKeeper in inaccessible!
Check your firewall configuration.
You can also check it with
zkCli.sh -server localhost:2181
There must have been some sort of connectivity problem.
I see you have it resolved now.
Next time you run into a situation like this, you should log onto the box that is having problems connecting and use telnet to see if you can connect.
eg: from your solr box:
telnet ec2-54-247-144-120.eu-west-1.compute.amazonaws.com 2181
and then try from the zk box too. It should start to illuminate where your issues are.
That eliminates any application layer issues and will tell you quite reliably wether or not you can connect. It you can't connect, then it's almost always some sort of security issue - either a firewall running somewhere (try - $service iptables stop) or it will be an issue with security group configuration in amazon.
The last potential problem is network availability. Despite what people think, the network is NOT reliable and should never be considered so. Anyone working in SOA/distributed systems will know this well :)
http://aphyr.com/posts/288-the-network-is-reliable
"A team from the University of Toronto and Microsoft Research studied the behavior of network failures in several of Microsoft’s datacenters. They found an average failure rate of 5.2 devices per day and 40.8 links per day with a median time to repair of approximately five minutes (and up to one week). "
While setting up SolrCloud and ZooKeeper I also ran into the "Error contacting service. It is probably not running." issue. The reason was a typo in a file name that ZooKeeper needs. The correct file name is "myid". I wrote "myip" by mistake. After the renaming of the file and restarting ZooKeeper (./zkServer.sh restart), my issue was resolved.
try to stop your solr instance solr.shutdown() so that you can create new CloudSolrServer instance for each thread

Resources