Failure to start Zeppelin - apache-zeppelin
When I start zeppelin o It starts on port 8080, it says port already in use. I tried changing port in zeppelin in config files, in zeppelin-site.xml.template made it too in zeppelin-env.sh.template adding the env line for the same port.
When I start zeppelin, I get OK, but if I see the open ports, It doesn't appear 8050 anywhere, so looks like It is still trying to deploy on port 8080
This is error I get when I click zeppelin.cmd
/zeppelin/interpreter/ignite/zeppelin-ignite_2.11-0.7.3.jar!/interpreter-settin
g.json
DEBUG [2018-05-06 07:33:49,111] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/jdbc/zeppelin-jdbc-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,126] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/kylin/zeppelin-kylin-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,281] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/lens/zeppelin-lens-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,431] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/livy/zeppelin-livy-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,466] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/md/zeppelin-markdown-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:49,754] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/pig/zeppelin-pig-0.7.3.jar!/interpreter-setting.json
WARN [2018-05-06 07:33:49,794] ({main} Interpreter.java[register]:406) - Static
initialization is deprecated for interpreter sql, You should change it to use i
nterpreter-setting.json in your jar or interpreter/{interpreter}/interpreter-set
ting.json
INFO [2018-05-06 07:33:49,794] ({main} InterpreterSettingManager.java[init]:305
) - Interpreter psql.sql found. class=org.apache.zeppelin.postgresql.PostgreSqlI
nterpreter
DEBUG [2018-05-06 07:33:52,889] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/python/zeppelin-python-0.7.3.jar!/interpreter-setting.jso
n
DEBUG [2018-05-06 07:33:53,234] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/sh/zeppelin-shell-0.7.3.jar!/interpreter-setting.json
DEBUG [2018-05-06 07:33:55,279] ({main} InterpreterSettingManager.java[registerI
nterpreterFromResource]:382) - Reading interpreter-setting.json from jar:file:/C
:/zeppelin/interpreter/spark/zeppelin-spark_2.11-0.7.3.jar!/interpreter-setting.
json
DEBUG [2018-05-06 07:33:55,284] ({main} InterpreterSettingManager.java[init]:320
) - Registered: psql.sql -> org.apache.zeppelin.postgresql.PostgreSqlInterpreter
. Properties: {postgresql.password={envName=null, propertyName=null, defaultValu
e=, description=The PostgreSQL user password}, postgresql.max.result={envName=nu
ll, propertyName=null, defaultValue=1000, description=Max number of SQL result t
o display.}, postgresql.user={envName=null, propertyName=null, defaultValue=gpad
min, description=The PostgreSQL user name}, postgresql.url={envName=null, proper
tyName=null, defaultValue=jdbc:postgresql://localhost:5432/, description=The URL
for PostgreSQL.}, postgresql.driver.name={envName=null, propertyName=null, defa
ultValue=org.postgresql.Driver, description= JDBC Driver Name}}
INFO [2018-05-06 07:33:55,289] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name ignite
INFO [2018-05-06 07:33:55,289] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name python
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name jdbc
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name psql
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name lens
INFO [2018-05-06 07:33:55,294] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name pig
INFO [2018-05-06 07:33:55,299] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name flink
INFO [2018-05-06 07:33:55,299] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name angular
INFO [2018-05-06 07:33:55,299] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name livy
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name file
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name elasticsearch
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name cassandra
INFO [2018-05-06 07:33:55,304] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name sh
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name spark
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name md
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name alluxio
INFO [2018-05-06 07:33:55,309] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name bigquery
INFO [2018-05-06 07:33:55,314] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name hbase
INFO [2018-05-06 07:33:55,314] ({main} InterpreterSettingManager.java[init]:337
) - InterpreterSettingRef name kylin
INFO [2018-05-06 07:33:55,450] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group md : id=2DEPH6PYC, name=md
INFO [2018-05-06 07:33:55,453] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group psql : id=2DD7ZJKPA, name=psql
INFO [2018-05-06 07:33:55,457] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group python : id=2DCAMH4GY, name=python
INFO [2018-05-06 07:33:55,463] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group file : id=2DDZH8WCB, name=file
INFO [2018-05-06 07:33:55,472] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group elasticsearch : id=2DE3BGT7H, name=elasticsearch
INFO [2018-05-06 07:33:55,476] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group bigquery : id=2DEVZXKAG, name=bigquery
INFO [2018-05-06 07:33:55,480] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group hbase : id=2DC4M79BE, name=hbase
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group sh : id=2DD1MA4VU, name=sh
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group pig : id=2DCNP1WD2, name=pig
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group alluxio : id=2DC7G4QW3, name=alluxio
INFO [2018-05-06 07:33:55,482] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group livy : id=2DEQ54GYT, name=livy
INFO [2018-05-06 07:33:55,487] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group flink : id=2DDWBXQYG, name=flink
INFO [2018-05-06 07:33:55,487] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group spark : id=2DF55VV2T, name=spark
INFO [2018-05-06 07:33:55,487] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group ignite : id=2DCJEAGTV, name=ignite
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group kylin : id=2DC5T3GZV, name=kylin
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group angular : id=2DFHJM4RD, name=angular
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group lens : id=2DC9V3E9V, name=lens
INFO [2018-05-06 07:33:55,492] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group jdbc : id=2DEECFK3N, name=jdbc
INFO [2018-05-06 07:33:55,497] ({main} InterpreterSettingManager.java[init]:366
) - InterpreterSetting group cassandra : id=2DCH1RQVX, name=cassandra
INFO [2018-05-06 07:33:55,537] ({main} InterpreterFactory.java[<init>]:130) - s
hiroEnabled: false
INFO [2018-05-06 07:33:55,858] ({main} VfsLog.java[info]:138) - Using "C:\Users
\Weltinfo\AppData\Local\Temp\vfs_cache" as temporary files store.
INFO [2018-05-06 07:33:56,055] ({main} GitNotebookRepo.java[<init>]:63) - Openi
ng a git repo at '/zeppelin/notebook'
INFO [2018-05-06 07:33:56,578] ({main} NotebookAuthorization.java[loadFromFile]
:96) - C:\zeppelin\conf\notebook-authorization.json
INFO [2018-05-06 07:33:56,581] ({main} Credentials.java[loadFromFile]:102) - C:
\zeppelin\conf\credentials.json
INFO [2018-05-06 07:33:56,710] ({main} StdSchedulerFactory.java[instantiate]:11
84) - Using default implementation for ThreadExecutor
INFO [2018-05-06 07:33:56,720] ({main} SimpleThreadPool.java[initialize]:268) -
Job execution threads will use class loader of thread: main
INFO [2018-05-06 07:33:56,775] ({main} SchedulerSignalerImpl.java[<init>]:61) -
Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignale
rImpl
INFO [2018-05-06 07:33:56,798] ({main} QuartzScheduler.java[<init>]:240) - Quar
tz Scheduler v.2.2.1 created.
INFO [2018-05-06 07:33:56,802] ({main} RAMJobStore.java[initialize]:155) - RAMJ
obStore initialized.
INFO [2018-05-06 07:33:56,806] ({main} QuartzScheduler.java[initialize]:305) -
Scheduler meta-data: Quartz Scheduler (v2.2.1) 'DefaultQuartzScheduler' with ins
tanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persis
tence. and is not clustered.
INFO [2018-05-06 07:33:56,810] ({main} StdSchedulerFactory.java[instantiate]:13
39) - Quartz scheduler 'DefaultQuartzScheduler' initialized from default resourc
e file in Quartz package: 'quartz.properties'
INFO [2018-05-06 07:33:56,813] ({main} StdSchedulerFactory.java[instantiate]:13
43) - Quartz scheduler version: 2.2.1
INFO [2018-05-06 07:33:56,815] ({main} QuartzScheduler.java[start]:575) - Sched
uler DefaultQuartzScheduler_$_NON_CLUSTERED started.
INFO [2018-05-06 07:33:57,388] ({main} FolderView.java[createFolder]:107) - Cre
ate folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,394] ({main} FolderView.java[createFolder]:107) - Cre
ate folder /
INFO [2018-05-06 07:33:57,396] ({main} Folder.java[setParent]:168) - Set parent
of / to /
INFO [2018-05-06 07:33:57,397] ({main} Folder.java[setParent]:168) - Set parent
of Zeppelin Tutorial to /
INFO [2018-05-06 07:33:57,398] ({main} Folder.java[addNote]:184) - Add note 2A9
4M5J1Z to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,456] ({main} Folder.java[addNote]:184) - Add note 2BW
JFTXKJ to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,504] ({main} Folder.java[addNote]:184) - Add note 2BY
EZ5EVK to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,528] ({main} Folder.java[addNote]:184) - Add note 2C2
AUG798 to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,557] ({main} Folder.java[addNote]:184) - Add note 2C3
5YU814 to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,574] ({main} Folder.java[addNote]:184) - Add note 2C5
7UKYWR to folder Zeppelin Tutorial
INFO [2018-05-06 07:33:57,575] ({main} Notebook.java[<init>]:127) - Notebook in
dexing started...
INFO [2018-05-06 07:33:58,305] ({main} LuceneSearch.java[addIndexDocs]:305) - I
ndexing 6 notebooks took 729ms
INFO [2018-05-06 07:33:58,305] ({main} Notebook.java[<init>]:129) - Notebook in
dexing finished: 6 indexed in 0s
INFO [2018-05-06 07:33:58,603] ({main} ServerImpl.java[initDestination]:94) - S
etting the server's publish address to be /
mai 06, 2018 7:33:58 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFOS: Scanning for root resource and provider classes in the packages:
org.apache.zeppelin.rest
mai 06, 2018 7:33:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasse
s
INFOS: Root resource classes found:
class org.apache.zeppelin.rest.SecurityRestApi
class org.apache.zeppelin.rest.HeliumRestApi
class org.apache.zeppelin.rest.ConfigurationsRestApi
class org.apache.zeppelin.rest.LoginRestApi
class org.apache.zeppelin.rest.InterpreterRestApi
class org.apache.zeppelin.rest.NotebookRepoRestApi
class org.apache.zeppelin.rest.CredentialRestApi
class org.apache.zeppelin.rest.NotebookRestApi
class org.apache.zeppelin.rest.ZeppelinRestApi
mai 06, 2018 7:33:58 AM com.sun.jersey.api.core.ScanningResourceConfig init
INFOS: No provider classes found.
mai 06, 2018 7:33:59 AM com.sun.jersey.server.impl.application.WebApplicationImp
l _initiate
INFOS: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012 05:14 PM'
mai 06, 2018 7:34:00 AM com.sun.jersey.spi.inject.Errors processErrorMessages
AVERTISSEMENT: The following warnings have been detected with resource and/or pr
ovider classes:
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.zeppel
in.rest.CredentialRestApi.getCredentials(java.lang.String) throws java.io.IOExce
ption,java.lang.IllegalArgumentException, should not consume any entity.
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.zeppel
in.rest.InterpreterRestApi.listInterpreter(java.lang.String), should not consume
any entity.
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.ze
ppelin.rest.NotebookRestApi.createNote(java.lang.String) throws java.io.IOExcept
ion, with URI template, "/", is treated as a resource method
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.ze
ppelin.rest.NotebookRestApi.getNoteList() throws java.io.IOException, with URI t
emplate, "/", is treated as a resource method
INFO [2018-05-06 07:34:00,216] ({main} ContextHandler.java[doStart]:744) - Star
ted o.e.j.w.WebAppContext#140e5a13{/,file:/C:/zeppelin/webapps/webapp/,AVAILABLE
}{C:\zeppelin\zeppelin-web-0.7.3.war}
WARN [2018-05-06 07:34:00,227] ({main} AbstractLifeCycle.java[setFailed]:212) -
FAILED ServerConnector#1317b708{HTTP/1.1}{0.0.0.0:8080}: java.net.BindException
: Address already in use: bind
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:32
1)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNet
workConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java
:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:18
9)
WARN [2018-05-06 07:34:00,233] ({main} AbstractLifeCycle.java[setFailed]:212) -
FAILED org.eclipse.jetty.server.Server#44a6a68e: java.net.BindException: Addres
s already in use: bind
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:32
1)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNet
workConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java
:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:18
9)
ERROR [2018-05-06 07:34:00,236] ({main} ZeppelinServer.java[main]:194) - Error w
hile running jettyServer
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:
223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:32
1)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNet
workConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java
:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLife
Cycle.java:68)
at org.apache.zeppelin.server.ZeppelinServer.main(ZeppelinServer.java:18
9)
"zeppelin-site.xml.template" and "zeppelin-env.sh.template" are not the files that are actual used when running Zeppelin. They are - as their name suggest - templates that should help to build the actual configuration files. They should not be changed.
Instead, create a copy of these files that are named "zeppelin-site.xml" or "zeppelin-env.sh" respectively and adopt these files according to your needs. During the next restart of Zeppelin the new settings should be picked up.
Related
APACHE zeppelin JSP support not found
I have a error relative to a JSP support not found log: INFO [2016-03-11 11:01:26,017] ({main} StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties' INFO [2016-03-11 11:01:26,018] ({main} StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version: 2.2.1 INFO [2016-03-11 11:01:26,019] ({main} QuartzScheduler.java[start]:575) - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started. INFO [2016-03-11 11:01:26,173] ({main} Notebook.java[]:107) - Notebook indexing started... INFO [2016-03-11 11:01:26,299] ({main} LuceneSearch.java[addIndexDocs]:285) - Indexing 0 notebooks took 125ms INFO [2016-03-11 11:01:26,313] ({main} Notebook.java[]:109) - Notebook indexing finished: 0 indexed in 0s INFO [2016-03-11 11:01:27,225] ({main} ServerImpl.java[initDestination]:94) - Setting the server's publish address to be / INFO [2016-03-11 11:01:28,112] ({main} StandardDescriptorProcessor.java[visitServlet]:284) - NO JSP Support for /srv/zeppelin/tmp/webapp, did not find org.apache.jasper.servlet.JspServlet INFO [2016-03-11 11:01:31,655] ({main} AbstractConnector.java[doStart]:338) - Started SelectChannelConnector#192.168.1.105:8090 INFO [2016-03-11 11:01:31,677] ({main} ZeppelinServer.java[main]:115) - Done, zeppelin server started zeppelin-site.xml <configuration> <property> <name>zeppelin.server.addr</name> <value>192.168.1.105</value> <description>Server address</description> </property> <property> <name>zeppelin.server.port</name> <value>8090</value> <description>Server port.</description> </property> <property> <name>zeppelin.server.context.path</name> <value>/srv/zeppelin/tmp/webapp</value> <description>Context Path of the Web Application</description> </property> <property> <name>zeppelin.war.tempdir</name> <value>/srv/zeppelin/tmp</value> <description>Location of jetty temporary directory</description> </property> <property> <name>zeppelin.notebook.dir</name> <value>/srv/zeppelin/notebook</value> <description>path or URI for notebook persist</description> </property> <property> <name>zeppelin.notebook.homescreen</name> <value></value> <description>id of notebook to be displayed in homescreen. ex) 2A94M5J1Z Empty value displays default home screen</description> </property> <property> <name>zeppelin.notebook.homescreen.hide</name> <value>false</value> <description>hide homescreen notebook from list when this value set to true</description> </property>
I solve the problem.... <property> <name>zeppelin.server.context.path</name> <value>/</value> <description>Context Path of the Web Application</description> </property> i think you have to put the root path and somewhere in the filesystem the software find the jsp support
Zeppelin disconnected
I downloaded zeppelin binary package and started it. However it's disconnected. Here is the log: INFO [2016-01-12 14:37:56,592] ({main} QuartzScheduler.java[initialize]:305) - Scheduler meta-data: Quartz Scheduler (v2.2.1) 'Defau ltQuartzScheduler' with instanceId 'NON_CLUSTERED' Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally. NOT STARTED. Currently in standby mode. Number of jobs executed: 0 Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads. Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered. INFO [2016-01-12 14:37:56,593] ({main} StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler 'DefaultQuartzScheduler' initi alized from default resource file in Quartz package: 'quartz.properties' INFO [2016-01-12 14:37:56,593] ({main} StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version: 2.2.1 INFO [2016-01-12 14:37:56,593] ({main} QuartzScheduler.java[start]:575) - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started. INFO [2016-01-12 14:37:56,805] ({main} ServerImpl.java[initDestination]:94) - Setting the server's publish address to be / INFO [2016-01-12 14:37:56,888] ({main} WebInfConfiguration.java[unpack]:478) - Extract jar:file:/data/users/huser/zeppelin/zeppelin- web-0.5.5-incubating.war!/ to /tmp/jetty-0.0.0.0-8080-zeppelin-web-0.5.5-incubating.war-_-any-/webapp INFO [2016-01-12 14:37:57,040] ({main} StandardDescriptorProcessor.java[visitServlet]:284) - NO JSP Support for /, did not find org. apache.jasper.servlet.JspServlet INFO [2016-01-12 14:37:57,941] ({main} AbstractConnector.java[doStart]:338) - Started SelectChannelConnector#0.0.0.0:8080 INFO [2016-01-12 14:37:57,941] ({main} ZeppelinServer.java[main]:108) - Started zeppelin server I searched and found that port+1 is the websocket port. However, it's not listening: netstat -na | grep 8080 tcp 0 0 :::8080 :::* LISTEN netstat -na | grep 8081 noting... And the web return the error: WebSocket connection to 'ws://ip:8080/ws' failed: Establishing a tunnel via proxy server failed. Can anyone help? Thanks.
Make sure port forwarding command executes on yr client system all the time you use zeppelin ssh -i ~/keys/mykey.pem -N -L 8122:xx.xxx.xxx.xxx:8890 hadoop#yy.yyy.yyy.yyy
SolrCloud - How to resolve "Could not find collection configName"
I am trying to start Solr in SolrCloud mode. I have created a new collection from collection1 and changed its name in file core.properties by setting the property name=logmail. But when I start Solr, I am getting the following error $ java -Dcollection.configName=logmail -DzkRun -Dnumshards=2 -DBootstrap_confdir=./solr/logmail/conf -jar start.jar 2165 [main] INFO org.apache.solr.common.cloud.ZkStateReader – Updating cluster state from ZooKeeper... 2179 [OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001] INFO org.apache.solr.cloud.Overseer – Starting to work on the main queue 2197 [main] INFO org.apache.solr.core.CoresLocator – Looking for core definitions underneath /home/rahul/Desktop/dev/solrcloud/solr 2203 [main] INFO org.apache.solr.core.CoresLocator – Found core logmail in /home/rahul/Desktop/dev/solrcloud/solr/logmail/ 2204 [main] INFO org.apache.solr.core.CoresLocator – Found core collection1 in /home/rahul/Desktop/dev/solrcloud/solr/collection1/ 2204 [main] INFO org.apache.solr.core.CoresLocator – Found 2 core definitions 2207 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – publishing core=logmail state=down collection=logmail 2207 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – publishing core=collection1 state=down collection=collection1 2208 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – numShards not found on descriptor - reading it from system property 2208 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – numShards not found on descriptor - reading it from system property 2214 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – look for our core node name 2214 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – waiting to find shard id in clusterstate for logmail 2214 [zkCallback-2-thread-1] INFO org.apache.solr.cloud.DistributedQueue – NodeChildrenChanged fired on path /overseer/queue state SyncConnected 2215 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Check for collection zkNode:logmail 2222 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – look for our core node name 2222 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Creating collection in ZooKeeper:logmail 2222 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – waiting to find shard id in clusterstate for collection1 2223 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Looking for collection configName 2223 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Check for collection zkNode:collection1 2224 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Creating collection in ZooKeeper:collection1 2224 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Looking for collection configName 2225 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 1 2226 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 1 2226 [OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001] INFO org.apache.solr.cloud.Overseer – Update state numShards=null message={ "core":"logmail", "roles":null, "base_url":"http://127.0.1.1:8983/solr", "node_name":"127.0.1.1:8983_solr", "state":"down", "shard":null, "collection":"logmail", "operation":"state"} 2226 [OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001] INFO org.apache.solr.cloud.Overseer – node=core_node1 is already registered 2227 [OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001] INFO org.apache.solr.cloud.Overseer – shard=shard1 is already registered 2255 [zkCallback-2-thread-1] INFO org.apache.solr.common.cloud.ZkStateReader – A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) 2268 [OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001] INFO org.apache.solr.cloud.Overseer – Update state numShards=null message={ "core":"collection1", "roles":null, "base_url":"http://127.0.1.1:8983/solr", "node_name":"127.0.1.1:8983_solr", "state":"down", "shard":null, "collection":"collection1", "operation":"state"} 2268 [OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001] INFO org.apache.solr.cloud.Overseer – node=core_node1 is already registered 2269 [OverseerStateUpdate-94955713964081152-127.0.1.1:8983_solr-n_0000000001] INFO org.apache.solr.cloud.Overseer – shard=shard1 is already registered 2288 [zkCallback-2-thread-1] INFO org.apache.solr.cloud.DistributedQueue – NodeChildrenChanged fired on path /overseer/queue state SyncConnected 2318 [zkCallback-2-thread-1] INFO org.apache.solr.common.cloud.ZkStateReader – A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) 5227 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 2 5228 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 2 8229 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 3 8229 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 3 11232 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 4 11232 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 4 14237 [coreLoadExecutor-6-thread-1] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 5 14237 [coreLoadExecutor-6-thread-2] INFO org.apache.solr.cloud.ZkController – Could not find collection configName - pausing for 3 seconds and trying again - try: 5 17237 [coreLoadExecutor-6-thread-1] ERROR org.apache.solr.cloud.ZkController – Could not find configName for collection logmail 17238 [coreLoadExecutor-6-thread-2] ERROR org.apache.solr.cloud.ZkController – Could not find configName for collection collection1 17240 [coreLoadExecutor-6-thread-1] ERROR org.apache.solr.core.CoreContainer – Error creating core [logmail]: Could not find configName for collection logmail found:null org.apache.solr.common.cloud.ZooKeeperException: Could not find configName for collection logmail found:null
It looks like there may be a discrepancy between what Solr has on the filesystem for collections based on your commands and what is in zookeeper. These are hard to fix; if possible I would recommend to delete your configuration files out of zookeeper and reload them.
You have a typo in your command. This should do the trick: $ java -Dcollection.configName=logmail -DzkRun -Dnumshards=2 -Dbootstrap_confdir=./solr/logmail/conf -jar start.jar
Loading solr configs in Cloudera SolrCloud
We try to import our data into SolrCloud using MapReduce batch indexing. We face a problem at the reduce phase, that solr.xml cannot be found. We create a 'twitter' collection but looking at the logs, after it failed to load in solr.xml, it uses the default one and tries to create 'collection1' (failed) and 'core1' (success) SolrCore. I'm not sure if we need to create our own solr.xml and where to put it (we try to put it at several places but it seems not to load in). Below is the log: 2022 [main] INFO org.apache.solr.hadoop.HeartBeater - Heart beat reporting class is org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl 2025 [main] INFO org.apache.solr.hadoop.SolrRecordWriter - Using this unpacked directory as solr home: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip 2025 [main] INFO org.apache.solr.hadoop.SolrRecordWriter - Creating embedded Solr server with solrHomeDir: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip, fs: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_-1828461666_1, ugi=nguyen (auth:SIMPLE)]], outputShardDir: hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014 2029 [Thread-64] INFO org.apache.solr.hadoop.HeartBeater - HeartBeat thread running 2030 [Thread-64] INFO org.apache.solr.hadoop.HeartBeater - Issuing heart beat for 1 threads 2083 [main] INFO org.apache.solr.core.SolrResourceLoader - new SolrResourceLoader for directory: '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/' 2259 [main] INFO org.apache.solr.hadoop.SolrRecordWriter - Constructed instance information solr.home /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip (/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip), instance dir /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/, conf dir /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/conf/, writing index to solr.data.dir hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data, with permdir hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014 2266 [main] INFO org.apache.solr.core.ConfigSolr - Loading container configuration from /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/solr.xml 2267 [main] INFO org.apache.solr.core.ConfigSolr - /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/solr.xml does not exist, using default configuration 2505 [main] INFO org.apache.solr.core.CoreContainer - New CoreContainer 696103669 2505 [main] INFO org.apache.solr.core.CoreContainer - Loading cores into CoreContainer [instanceDir=/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/] 2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting socketTimeout to: 0 2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting urlScheme to: http:// 2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting connTimeout to: 0 2515 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting maxConnectionsPerHost to: 20 2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting corePoolSize to: 0 2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting maximumPoolSize to: 2147483647 2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting maxThreadIdleTime to: 5 2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting sizeOfQueue to: -1 2516 [main] INFO org.apache.solr.handler.component.HttpShardHandlerFactory - Setting fairnessPolicy to: false 2527 [main] INFO org.apache.solr.client.solrj.impl.HttpClientUtil - Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false 2648 [main] INFO org.apache.solr.logging.LogWatcher - Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)] 2676 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.CoreContainer - Creating SolrCore 'collection1' using instanceDir: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1 2677 [coreLoadExecutor-3-thread-1] INFO org.apache.solr.core.SolrResourceLoader - new SolrResourceLoader for directory: '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/' 2691 [coreLoadExecutor-3-thread-1] ERROR org.apache.solr.core.CoreContainer - Failed to load file /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/solrconfig.xml 2693 [coreLoadExecutor-3-thread-1] ERROR org.apache.solr.core.CoreContainer - Unable to create core: collection1 org.apache.solr.common.SolrException: Could not load config for solrconfig.xml at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:596) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:661) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:368) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:360) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/conf/', cwd=/data/05/mapred/local/taskTracker/nguyen/jobcache/job_201311191613_0320/attempt_201311191613_0320_r_000014_0/work at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:322) at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:287) at org.apache.solr.core.Config.<init>(Config.java:116) at org.apache.solr.core.Config.<init>(Config.java:86) at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:120) at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:593) ... 11 more 2695 [coreLoadExecutor-3-thread-1] ERROR org.apache.solr.core.CoreContainer - null:org.apache.solr.common.SolrException: Unable to create core: collection1 at org.apache.solr.core.CoreContainer.recordAndThrow(CoreContainer.java:1158) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:670) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:368) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:360) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Caused by: org.apache.solr.common.SolrException: Could not load config for solrconfig.xml at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:596) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:661) ... 10 more Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in classpath or '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/conf/', cwd=/data/05/mapred/local/taskTracker/nguyen/jobcache/job_201311191613_0320/attempt_201311191613_0320_r_000014_0/work at org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:322) at org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:287) at org.apache.solr.core.Config.<init>(Config.java:116) at org.apache.solr.core.Config.<init>(Config.java:86) at org.apache.solr.core.SolrConfig.<init>(SolrConfig.java:120) at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:593) ... 11 more 2697 [main] INFO org.apache.solr.core.CoreContainer - Creating SolrCore 'core1' using instanceDir: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip 2697 [main] INFO org.apache.solr.core.SolrResourceLoader - new SolrResourceLoader for directory: '/data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/' 2751 [main] INFO org.apache.solr.core.SolrConfig - Adding specified lib dirs to ClassLoader 2752 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/extraction/lib (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/extraction/lib). 2752 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist). 2752 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/clustering/lib/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/clustering/lib). 2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist). 2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/langid/lib/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/langid/lib). 2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist). 2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../contrib/velocity/lib (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../contrib/velocity/lib). 2753 [main] WARN org.apache.solr.core.SolrResourceLoader - Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/../../../dist). 2785 [main] INFO org.apache.solr.update.SolrIndexConfig - IndexWriter infoStream solr logging is enabled 2790 [main] INFO org.apache.solr.core.SolrConfig - Using Lucene MatchVersion: LUCENE_44 2869 [main] INFO org.apache.solr.core.Config - Loaded SolrConfig: solrconfig.xml 2879 [main] INFO org.apache.solr.schema.IndexSchema - Reading Solr Schema from schema.xml 2937 [main] INFO org.apache.solr.schema.IndexSchema - [core1] Schema name=twitter 3352 [main] INFO org.apache.solr.schema.IndexSchema - unique key field: id 3471 [main] INFO org.apache.solr.schema.FileExchangeRateProvider - Reloading exchange rates from file currency.xml 3478 [main] INFO org.apache.solr.schema.FileExchangeRateProvider - Reloading exchange rates from file currency.xml 3635 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - Solr Kerberos Authentication disabled 3636 [main] INFO org.apache.solr.core.JmxMonitoredMap - No JMX servers found, not exposing Solr information with JMX. 3652 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - creating directory factory for path hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data 3686 [main] INFO org.apache.solr.core.CachingDirectoryFactory - return new directory for hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data 3711 [main] WARN org.apache.solr.core.SolrCore - [core1] Solr index directory 'hdfs:/master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data/index' doesn't exist. Creating new index... 3719 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - creating directory factory for path hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data/index 3719 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - Number of slabs of block cache [1] with direct memory allocation set to [true] 3720 [main] INFO org.apache.solr.core.HdfsDirectoryFactory - Block cache target memory usage, slab size of [134217728] will allocate [1] slabs and use ~[134217728] bytes 3721 [main] INFO org.apache.solr.store.blockcache.BufferStore - Initializing the 1024 buffers with [8192] buffers. 3740 [main] INFO org.apache.solr.store.blockcache.BufferStore - Initializing the 8192 buffers with [8192] buffers. 3891 [main] INFO org.apache.solr.core.CachingDirectoryFactory - return new directory for hdfs://master.hadoop:8020/user/nguyen/twitter/outdir/reducers/_temporary/_attempt_201311191613_0320_r_000014_0/part-r-00014/data/index 3988 [main] INFO org.apache.solr.update.LoggingInfoStream - [IFD][main]: init: current segments file is "null"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper#65b01d5d 3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IFD][main]: now checkpoint "" [0 segments ; isCommit = false] 3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IFD][main]: 0 msec to checkpoint 3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IW][main]: init: create=true 3992 [main] INFO org.apache.solr.update.LoggingInfoStream - [IW][main]: dir=NRTCachingDirectory(org.apache.solr.store.hdfs.HdfsDirectory#17e5a6d8 lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory#7f117668; maxCacheMB=192.0 maxMergeSizeMB=16.0)
solr looks for solr.home parameter and searchs solrConfig.xml file there. if there is none it tries to load default configuration. it looks like your solr home is /data/06/mapred/local/taskTracker/distcache/3866561797898787678_-1754062477_512745567/master.hadoop/tmp/9501daf9-5011-4665-bae3-d5af1c8bcd62.solr.zip/collection1/ check that folder for solrconfig.xml file if there is none, copy one from example directory of solr if there is one, match the file/folder permissions with the server instance
zookeeper does not run?
I wanted to run a solr cloud with solr 4.3.0. (I am using aws ubuntu-12.04-lts micro instances) So I followed this toturial: which basically says, start the zookeeper and connect the solr instances to it. Here's how I start the zookeeper. First I copied the config like described in the tutorial sudo cp zookeeper-3.4.5/conf/zoo_sample.cfg zookeeper-3.4.5/conf/zoo.cfg Then I started the zookeeper ubuntu#ip-10-48-159-36:/opt$ sudo zookeeper-3.4.5/bin/zkServer.sh start JMX enabled by default Using config: /opt/zookeeper-3.4.5/bin/../conf/zoo.cfg Starting zookeeper ... STARTED Looks fine so far. I checked the status: ubuntu#ip-10-48-159-36:/opt$ sudo zookeeper-3.4.5/bin/zkServer.sh status JMX enabled by default Using config: /opt/zookeeper-3.4.5/bin/../conf/zoo.cfg Error contacting service. It is probably not running. Which seems a bit weird already. If I try to connect with the client (remote as well as local), its seems to work ubuntu#ip-10-234-223-69:/opt$ zookeeper-3.4.5/bin/zkCli.sh -server ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 Connecting to ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 2013-06-07 11:07:01,996 [myid:] - INFO [main:Environment#100] - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 2013-06-07 11:07:02,000 [myid:] - INFO [main:Environment#100] - Client environment:host.name=ip-10-234-223-69.eu-west-1.compute.internal 2013-06-07 11:07:02,000 [myid:] - INFO [main:Environment#100] - Client environment:java.version=1.6.0_27 2013-06-07 11:07:02,002 [myid:] - INFO [main:Environment#100] - Client environment:java.vendor=Sun Microsystems Inc. 2013-06-07 11:07:02,003 [myid:] - INFO [main:Environment#100] - Client environment:java.home=/usr/lib/jvm/java-6-openjdk-amd64/jre 2013-06-07 11:07:02,003 [myid:] - INFO [main:Environment#100] - Client environment:java.class.path=/opt/zookeeper-3.4.5/bin/../build/classes:/opt/zookeeper-3.4.5/bin/../build/lib/*.jar:/opt/zookeeper-3.4.5/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper-3.4.5/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper-3.4.5/bin/../lib/netty-3.2.2.Final.jar:/opt/zookeeper-3.4.5/bin/../lib/log4j-1.2.15.jar:/opt/zookeeper-3.4.5/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.5/bin/../zookeeper-3.4.5.jar:/opt/zookeeper-3.4.5/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.5/bin/../conf: 2013-06-07 11:07:02,004 [myid:] - INFO [main:Environment#100] - Client environment:java.library.path=/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib 2013-06-07 11:07:02,008 [myid:] - INFO [main:Environment#100] - Client environment:java.io.tmpdir=/tmp 2013-06-07 11:07:02,009 [myid:] - INFO [main:Environment#100] - Client environment:java.compiler=<NA> 2013-06-07 11:07:02,018 [myid:] - INFO [main:Environment#100] - Client environment:os.name=Linux 2013-06-07 11:07:02,019 [myid:] - INFO [main:Environment#100] - Client environment:os.arch=amd64 2013-06-07 11:07:02,019 [myid:] - INFO [main:Environment#100] - Client environment:os.version=3.2.0-40-virtual 2013-06-07 11:07:02,020 [myid:] - INFO [main:Environment#100] - Client environment:user.name=ubuntu 2013-06-07 11:07:02,020 [myid:] - INFO [main:Environment#100] - Client environment:user.home=/home/ubuntu 2013-06-07 11:07:02,021 [myid:] - INFO [main:Environment#100] - Client environment:user.dir=/opt 2013-06-07 11:07:02,029 [myid:] - INFO [main:ZooKeeper#438] - Initiating client connection, connectString=ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher#182d9c06 Welcome to ZooKeeper! 2013-06-07 11:07:02,074 [myid:] - INFO [main-SendThread(ip-10-48-159-36.eu-west-1.compute.internal:2181):ClientCnxn$SendThread#966] - Opening socket connection to server ip-10-48-159-36.eu-west-1.compute.internal/10.48.159.36:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled [zk: ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181(CONNECTING) 0] 2013-06-07 11:07:32,100 [myid:] - INFO [main-SendThread(ip-10-48-159-36.eu-west-1.compute.internal:2181):ClientCnxn$SendThread#1083] - Client session timed out, have not heard from server in 30038ms for sessionid 0x0, closing socket connection and attempting reconnect 2013-06-07 11:07:33,204 [myid:] - INFO [main-SendThread(ip-10-48-159-36.eu-west-1.compute.internal:2181):ClientCnxn$SendThread#966] - Opening socket connection to server ip-10-48-159-36.eu-west-1.compute.internal/10.48.159.36:2181. Will not attempt to authenticate using SASL (unknown error) Now I tried to connect a solr instance to it. In the web interface of tomcat7 it only tells me "503 - Server is shutting down", so I checked the solr logs 2013-06-07 11:16:36,065 [pool-2-thread-1] INFO org.apache.solr.servlet.SolrDispatchFilter . SolrDispatchFilter.init() 2013-06-07 11:16:36,100 [pool-2-thread-1] INFO org.apache.solr.core.SolrResourceLoader . Using JNDI solr.home: /opt/solr-4.3.0/example/solr 2013-06-07 11:16:36,132 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . looking for solr config file: /opt/solr-4.3.0/example/solr/solr.xml 2013-06-07 11:16:36,138 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . New CoreContainer 1285984216 2013-06-07 11:16:36,146 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . Loading CoreContainer using Solr Home: '/opt/solr-4.3.0/example/solr/' 2013-06-07 11:16:36,152 [pool-2-thread-1] INFO org.apache.solr.core.SolrResourceLoader . new SolrResourceLoader for directory: '/opt/solr-4.3.0/example/solr/' 2013-06-07 11:16:36,567 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting socketTimeout to: 0 2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting urlScheme to: http:// 2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting connTimeout to: 0 2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting maxConnectionsPerHost to: 20 2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting corePoolSize to: 0 2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting maximumPoolSize to: 2147483647 2013-06-07 11:16:36,568 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting maxThreadIdleTime to: 5 2013-06-07 11:16:36,569 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting sizeOfQueue to: -1 2013-06-07 11:16:36,569 [pool-2-thread-1] INFO org.apache.solr.handler.component.HttpShardHandlerFactory . Setting fairnessPolicy to: false 2013-06-07 11:16:36,578 [pool-2-thread-1] INFO org.apache.solr.client.solrj.impl.HttpClientUtil . Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false 2013-06-07 11:16:36,879 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . Registering Log Listener 2013-06-07 11:16:36,881 [pool-2-thread-1] INFO org.apache.solr.core.CoreContainer . Zookeeper client=ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 2013-06-07 11:16:36,888 [pool-2-thread-1] INFO org.apache.solr.client.solrj.impl.HttpClientUtil . Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=0&connTimeout=0 2013-06-07 11:16:37,040 [pool-2-thread-1] INFO org.apache.solr.common.cloud.ConnectionManager . Waiting for client to connect to ZooKeeper 2013-06-07 11:16:52,046 [pool-2-thread-1] ERROR org.apache.solr.servlet.SolrDispatchFilter . Could not start Solr. Check solr/home property and the logs 2013-06-07 11:16:52,103 [pool-2-thread-1] ERROR org.apache.solr.core.SolrCore . null:java.lang.RuntimeException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 within 15000 ms at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:130) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:88) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:170) at org.apache.solr.core.CoreContainer.initZooKeeper(CoreContainer.java:242) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:495) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:358) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:326) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:124) at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:277) at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:258) at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:382) at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:103) at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4638) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5294) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:895) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:871) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615) at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649) at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper ec2-54-247-144-120.eu-west-1.compute.amazonaws.com:2181 within 15000 ms at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:173) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:127) ... 25 more 2013-06-07 11:16:52,104 [pool-2-thread-1] INFO org.apache.solr.servlet.SolrDispatchFilter . SolrDispatchFilter.init() done What does it tell me? On the same instance I just connected with the client successfully... :( So where is the problem? [Edit:] Instead of using amazons ec**.amazon.* address I used the network addresses 10.X.X.X for telling solr where the zookeeper is. It seems to work.
You have your answer - Your ZooKeeper in inaccessible! Check your firewall configuration. You can also check it with zkCli.sh -server localhost:2181
There must have been some sort of connectivity problem. I see you have it resolved now. Next time you run into a situation like this, you should log onto the box that is having problems connecting and use telnet to see if you can connect. eg: from your solr box: telnet ec2-54-247-144-120.eu-west-1.compute.amazonaws.com 2181 and then try from the zk box too. It should start to illuminate where your issues are. That eliminates any application layer issues and will tell you quite reliably wether or not you can connect. It you can't connect, then it's almost always some sort of security issue - either a firewall running somewhere (try - $service iptables stop) or it will be an issue with security group configuration in amazon. The last potential problem is network availability. Despite what people think, the network is NOT reliable and should never be considered so. Anyone working in SOA/distributed systems will know this well :) http://aphyr.com/posts/288-the-network-is-reliable "A team from the University of Toronto and Microsoft Research studied the behavior of network failures in several of Microsoft’s datacenters. They found an average failure rate of 5.2 devices per day and 40.8 links per day with a median time to repair of approximately five minutes (and up to one week). "
While setting up SolrCloud and ZooKeeper I also ran into the "Error contacting service. It is probably not running." issue. The reason was a typo in a file name that ZooKeeper needs. The correct file name is "myid". I wrote "myip" by mistake. After the renaming of the file and restarting ZooKeeper (./zkServer.sh restart), my issue was resolved.
try to stop your solr instance solr.shutdown() so that you can create new CloudSolrServer instance for each thread