I have cluster of 7 nodes (6 solr + 1 zookeper) in my local system setup. My used configuration is solr 5.5.0 and zookeeper-3.4.8. Actually I am having a testing environment in local before moving to production environment. For that I have created 6 instances of solr as well as Zookeeper (1 instance of each on a laptop) and 1 instance of zookeeper especially on a laptop. Now when I test it in local using soap request My 1 node's tlog have unexceptional size at same time rest are fine.
what can be issue, how should I approach to rectify. I am not able to understand what is problem because my rest nodes are fine except one node.
Note:- Every machine(laptop) have same solrconfig.xml.
Solr Cloud always starts in example and configuration specified in solr.in.sh is not working.
What are changes required to start with custom configuration ?
I have setup Zookeeper in ensemble mode and also do changes in solr.in.sh. But, it seems Solr is always starts with Example
I saw your answer in comments, yet completing the answer for others.
To start solr in cloud mode you need to add "-cloud":- solr start -cloud
-e start in interactive mode with one example. This is just for newbies to get started with solrcloud
NOTE: I've tried everything in the comments below and everything else I can think of. At this point I have to assume there's a bug of some kind and that a restart will NOT bring SOLR up in cloud mode unless you roll your own init.d stuff.
==================================================
I have 3 SOLR nodes and 3 Zookeeper nodes.
The SOLR Nodes are SOLR 5.4 on Ubuntu 14 and were installed based on the instructions here:
https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production
If I issue this command to start or restart SOLR on the command line, everything looks fine in the SOLR Admin UI and all my nodes are green in the UI
sudo /opt/solr/bin/solr restart -c -z 192.168.56.5,192.168.56.6,192.168.56.7/solr5_4
However, even though I have a ZK_HOST entry in my solr.in.sh I cannot get the nodes to show up in the SOLR Admin console correctly if I try:
service solr restart
Or if I reboot the VM.
My ZK_HOST entry in solr.in.sh looks like this:
ZK_HOST="192.168.56.5,192.168.56.6,192.168.56.7/solr5_4"
I also tried it this way (no quotes, just in case) because that's how it looks on the Apache wiki page I was reading:
ZK_HOST=192.168.56.5,192.168.56.6,192.168.56.7/solr5_4
I always have to run the command line to get the SOLR instances to show up correctly in the Admin UI. It would be preferrable to have this "just happen" when rebooting the VM.
If I run service solr restart on any of them, they show as "down" in the Admin UI and the core I am using disappears from the Admin UI for the one IP address I'm looking at.
Why is this and what settings are required to get SOLR to start on boot into "Cloud Mode" with the correct Zookeeper settings?
Until a recent change, the docs for setting SOLR up for Prod had a slight misdirection. The bottom line here is that /etc/default/solr.in.sh was what controlled the SOLR configs on startup. NOT the one mentioned in the docs (which was somewhere else anyway /opt/solr/bin)
Once I added the ZKHOST setting in /etc/default/solr.in.sh and restarted the service (or rebooted the server) SOLR came up in "Cloud" mode every time.
I have a solr cloud cluster with three different machines. Initially when there was only one machine I had enabled replication handler for master/slave in solrconfig.xml. But then I changed the config and commented out the replication handler part. Then I add the other two machines to the cluster and also created a cluster of zookeepers(one on each machine) and uploaded the new solrconfig.xml file.
But I still see the master/slave setup on the initial machine( that one that existed from beginning) and not on the other two machines. They all have the same config now, so why is the first machine still showing the master/slave part. Do I need to reload the zookeeper for that machine or something?
Any help will be appreciated.
thanks.
I have been trying to implement a SolrCloud, and everything works fine until I try to create a collection with 6 shards. My setup is as follows:
5 virtual servers, all running Ubuntu 14.04, hosted by a single company across different data centers
3 servers running ZooKeeper 3.4.6 for the ensemble
2 servers, each running Solr 5.1.0 server (Jetty)
The Solr instances each have a 2TB, ext4 secondary disk for the indexes, mounted at /solrData/Indexes. I set this value in solrconfig.xml via <dataDir>/solrData/Indexes</dataDir>, and uploaded it to the ZooKeeper ensemble. Note that these secondary disks are neither NAS nor NFS, which I know can cause problems. The solr user owns /solrData.
All the intra-server communication is via private IP, since all are hosted by the same company. I'm using iptables for firewall, and the ports are open and all the servers are communicating successfully. Config upload to ZooKeeper is successful, and I can see via the Solr admin interface that both nodes are available.
The trouble starts when I try to create a collection using the following command:
http://xxx.xxx.xxx.xxx:8983/solr/admin/collections?action=CREATE&name=coll1&maxShardsPerNode=6&router.name=implicit&shards=shard1,shard2,shard3,shard4,shard5,shard6&router.field=shard&async=4444
Via the Solr UI logging, I see that multiple index creation commands are issued simultaneously, like so:
6/25/2015, 7:55:45 AM WARN SolrCore [coll1_shard2_replica1] Solr index directory '/solrData/Indexes/index' doesn't exist. Creating new index...
6/25/2015, 7:55:45 AM WARN SolrCore [coll1_shard1_replica2] Solr index directory '/solrData/Indexes/index' doesn't exist. Creating new index...
Ultimately the task gets reported as complete, but in the log, I have locking errors:
Error creating core [coll1_shard2_replica1]: Lock obtain timed out: SimpleFSLock#/solrData/Indexes/index/write.lock
SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
Error closing IndexWriter
If I look at the cloud graph, maybe a couple of the shards will have been created, others are closed or recovering, and if I restart Solr, none of the cores can fire up.
Now, I know what you're going to say: follow this SO post and change solrconfig.xml locking settings to this:
<unlockOnStartup>true</unlockOnStartup>
<lockType>simple</lockType>
I did that, and it had no impact whatsoever. Hence the question. I'm about to have to release a single Solr instance into production, which I hate to do. Does anybody know how to fix this?
Based on the log entry you supplied, it looks like Solr may be creating the data (index) directory for EACH shard in the same folder.
Solr index directory '/solrData/Indexes/index' doesn't exist. Creating new index...
This message was shown for two different collections and it references the same location. What I usually do, is change my Solr Home to a different directory, under which all collection "instance" stuff will be created. Then I manually edit the core.properties for each shard to specify the location of the index data.