I am trying to setup Solr cloud with multiple instance(s), all on one machine my zookeeper ensemble is runing on local machine.
for first instance
bin/solr start -c -s $INSATNCE_DIR -p $solrport -z localhost:2181,localhost:2182,localhost:2183 -Dsolr.log=/logs -Dsolr.install.dir=/solr-5.4.0 -V -Dbootstrap_conf=true
and for other instance
bin/solr start -c -s $INSATNCE_DIR -p $solrport -z localhost:2181,localhost:2182,localhost:2183 -Dsolr.log=/logs -Dsolr.install.dir=/solr-5.4.0 -V -Dbootstrap_conf=false
all works fine, but each of these instance admin interface under cloud>Tree only lists one live node
localhost:8983_solr
not the other instances.
This is causing issue when i am creating collection to be distributed to nodeSet.
Anyone has any idea whats am i doing wrong?
Related
I use Solr Cloud for my project and I always start it with bin/solr start -e -cloud that always prompt all the questions for making a new collection or re-use one.
But is there a command that can directly start the collection I want, like bin/solr start [collectionName] -cloud -p 8983?
I didn't find anything in the Solr Manual.
Thanks :)
All collections are available by default, so starting Solr in cloud mode should be enough.
bin/solr start -c
The default port is 8983, so there is no need to give the -p parameter unless you're changing it.
I have to do the following:
I have to deploy Solr on 2 servers/nodes.
Deploy Zookeeper on another server.
Upload a custom config to Zookeeper
Create a custom collection with 2 shards and 2 replicas
Version of Solr 7.4.0 & Zookeeper: 3.4.12
I have done the following:
Set Up Zookeeper:
Created a Zookeeper data folder & made a zoo.conf & put the dataDir there.
Started zookeeper using ./zkServer.sh start
Set up Solr:
Started Solr using:
./solr start -cloud -s /home/demo/LocalFolder/Downloads/SolrHome -p 8987 -z localhost:2181
Trying to upload config in Zookeeper using:
./solr create -c mycollection -d /media/sf_VM/Dump/conf
It is giving me an exception:
Caused by: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
... 17 more
</pre>
I have searched many pages & seen Solr tutorials but there they have used the default examples. I just dont have any step by step idea to
How to Upload a config in Zookeeper?
Then what I need to do to create a collection pointing to that config. I want that collection to have 2 shards & 2 replicas.
Where will be the solr.xml. If it should be in Zookeeper how do I upload it there
How do I see in Zookeeper that the config has been uploaded?
I know this question might be a duplicate. I have read several of posts but not able to come up a solution. Please help.
Well I figured out how to setup. Please note that I have read these steps from different site & gathered here step by step:
1. Set Up ZooKeeper:
This is used to keep the collection specific configuration files to store in a central space & map it with a name. Later we can use this name to create collection which would point to this config. This config is the same as you find in the below folder:
solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf
1.1 Download latest Zookeeper. I have used 3.4.12
1.2 Unpack downloaded archive and copy conf/zoo_sample.cfg to conf/zoo.cfg
1.3 Modify zoo.cfg:
1.3.1 Change dataDir to directory where you want to hold all cluster configuration data.
dataDir=/var/zookeeper/data
1.3.2 Add information about all Zookeeper servers: I have used only 1 ZooKeeper server so this is not required. If you want to add more servers then please go through the below links:
DZone Tutorial
Apache Tutorial
1.3.3 Start ZooKeeper using (after going to zookeeper-3.4.12/):
./bin/zkServer.sh start-foreground conf/zoo.cfg
OR
./bin/zkServer.sh start conf/zoo.cfg
Note: You can stop the ZooKeeper using the below command:
bin/zkServer.sh stop
1.3.4 ZooKeeper Status:
Hit the below:
bin/zkServer.sh status
Or do telnet localhost 2181 and hit stats when connected.
2. Set Up Solr
2.1 Download the Solr
2.2 Extract install_solr_service.sh from the .tar file. Solr includes a service installation script (bin/install_solr_service.sh) to help you install Solr as a service on Linux. For more info click here.
tar -xzf solr-7.4.0.tgz solr-7.4.0/bin/install_solr_service.sh --strip-components=2
2.3 Install Solr as a service using the above script:
sudo bash ./install_solr_service.sh solr-7.4.0.tgz
This will also extracting solr-7.4.0.tgz to /opt/solr
2.4 Go to /opt/solr and do the following:
mkdir solr/server/solr2
mkdir solr/server/solr3
mkdir solr/server/solr4
cp solr/server/solr/solr.xml solr/server/solr2
cp solr/server/solr/solr.xml solr/server/solr3
cp solr/server/solr/solr.xml solr/server/solr4
2.5 Change the jetty port in solr.xml. Do this for all the 3 solr.xml mentioned in the above step:
vi solr/server/solr2/solr.xml
Search for the port 8983 & change it 8984 (for solr2), change it 8985 (for solr3), change it 8986 (for solr4)
2.6 Stop the Solr running at 8983
root#dev-base:/opt/solr/bin# ./solr stop -p 8983
2.7 Start all the solr instances:
root#dev-base:/opt/solr# bin/solr start -c -s server/solr -p 8983 -z localhost:2181 -noprompt -force
root#dev-base:/opt/solr# bin/solr start -c -s server/solr2 -p 8984 -z localhost:2181 -noprompt -force
root#dev-base:/opt/solr# bin/solr start -c -s server/solr3 -p 8985 -z localhost:2181 -noprompt -force
root#dev-base:/opt/solr# bin/solr start -c -s server/solr4 -p 8986 -z localhost:2181 -noprompt -force
Note: Running solr as a root is not recommended for security reason.
2.8 See Solr Status:
root#dev-base:/opt/solr# bin/solr status
3 Make a Custom Config
3.1 Copy the conf directory from solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf to another location (In my case it is /media/sf_VM/Dump/new/conf ).
3.2 Change the managed-schema file inside conf to specify the fields you are using.
4 Upload the config to ZooKeeper:
root#dev-base:/opt/solr# bin/solr zk -z localhost:2181 upconfig -d /media/sf_VM/Dump/new/conf -n myConf6
The name of the config I have uploaded is myConf6
5 Create a Solr Collection using this custom config
root#dev-base:/opt/solr# bin/solr create -c myNewCollection -n myConf6 -shards 2 -replicationFactor 2 -force
Hit Solr Admin URL
6 Index Data Using POST API using Json
URL: http://localhost:8983/solr/myNewCollection/update
Method: POST
Body:
[{
"_id": "99999999999999999999",
"author": [
"New Inserted 9000"
],
"authorLastName": [
"New Inserted 9000"
],
"impn": "New Inserted 9000",
"isbn10": "9999999999",
"isbn13": "9999999999999",
"title": "New Inserted 9000",
"publisher": "New Inserted 9000",
"sales_a": 5.0,
"sales_t": 5.0,
"haveImage": 1,
"pages": "76",
"image": "http://ip.ip.com/is/image/",
"format": "Paper",
"mtc_id": "99999999999",
"subjects" : [
"9000"
]
"rating": 0,
"description_long": "Snahashis call me in your marriage."
}
Delete a configuration in Zookeeper:
If you want to delete an old/wrong configuration already uploaded to ZooKeeper run the below command:
./server/scripts/cloud-scripts/zkcli.sh -cmd clear -z "<ZK_HOST>:<ZK_PORT>" /configs/AAA
The path of the configuration is /configs/< name of the configset >
To delete a specific file:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd clear /configs/<MY_COLLECTION>/solrconfig.xml
Upload an updated file:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd putfile /configs/<MY_COLLECTION>/solrconfig.xml /<MY_UPDATED_FILE_LOCAL_FOLDER>/solrconfig.xml
Then we need to restart the solr nodes.
To Delete a Collection Via API:
First delete alias created on this collection (if any)
http://localhost:8983/solr/admin/collections?action=DELETEALIAS&name=aliasName
Delete Collection:
http://localhost:8983/solr/admin/collections?action=DELETE&name=collectionName
I'm trying to activate nutch and solr as dockers out of the box to consume that in REST.
Tried
same docker - started it with
docker run --name nutch_solr -d -p 8899:8899 -p 8983:8983 -t momer/docker-solr-nutch:4.6.1
can't surf to ..:8983, or ...:8899
Tried different dockers - nutch and solr
started it with:
docker run --name my_nutch -d -p 8899:8899 -e SOLRURL=192.168.99.100:8983 -t meabed/nutch
Solr is up and running, but can't really work with nutch (my question)
tried more another nutch docker - what am I missing? I'd like to write a post on how to raise solr and nutch dockers in 5 minutes, but I just can't seem to work with it..
so, does someone know how to activate it including send a sample of crawling job for work?
I have started learning solr.I have downloaded the latest zip(5.1.0) provided by solr and run the server using bin/solr start -e cloud -noprompt.
I check that this internally calls
bin/solr start -cloud -s example/cloud/node1/solr -p 8983
bin/solr start -cloud -s example/cloud/node2/solr -p 7574 -z localhost:9983
I check that these is no config(conf/solrconfig.xml) defined in example/cloud/node1/solr so how does solr load config from the SOLR_HOME/configsets directory?
I read the documentation on several places but i am still unable to figure out the use of cloud like in 'bin/solr start -cloud -s ... ' and use of zookeeper.
Please help.
when you are working on solr cloud with zookeeper, you have to upload your solr config on zookeeper.
./bin/solr zk -upconfig -z localhost:2181,localhost:2182,localhost:2182 -n my-config -d server/solr/files/conf/
using upconfig you can upload your solr config, only have to provide path of your config directory.
You can use config name(my-config) for create core using api.
http://XXX.XXX.XXX.XXX:8983/solr/admin/collections?action=CREATE&name=irTest&numShards=3&replicationFactor=2&maxShardsPerNode=3&collection.configName=my-config
So it will create core using your config only.
Download the latest version of the Apache solr reference guide.
https://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide/apache-solr-ref-guide-5.1.pdf
check this section in the PDF.
Configuration Directories and SolrCloud
Since you are not specifying a specific configset, the default is loaded.
First, if you don't provide the -d or -n options, then the default
configuration ($SOLR_HOME/server/solr/con
figsets/data_driven_schema_configs/conf) is uploaded to ZooKeeper
using the same name as the collection
I have 3 Zookeeper servers running at server1:2181, server2:2181 and server3:2181.
I want to start 4 Solr servers at server1:8983,server2:8983,server3:8983 and server4:8983 to point to Zookeeper Ensemble above. So at server1, I run a command:
bin>solr -c -z server1:2181,server2:2181,server3:2181 -m 2g
and I received an error message:
Missing operand.
Invalid command-line option: server2:2181
Usage:.........
but if I point to one Zookeeper server such as:
bin>solr -c -z server1:2181 -m 2g
it starts successfully.
All server running in Windows.
What did I do wrong? Or does the Solr start script in Windows have an error?
You need to put your zk connection string in quotes.