I have a solr cloud cluster with three different machines. Initially when there was only one machine I had enabled replication handler for master/slave in solrconfig.xml. But then I changed the config and commented out the replication handler part. Then I add the other two machines to the cluster and also created a cluster of zookeepers(one on each machine) and uploaded the new solrconfig.xml file.
But I still see the master/slave setup on the initial machine( that one that existed from beginning) and not on the other two machines. They all have the same config now, so why is the first machine still showing the master/slave part. Do I need to reload the zookeeper for that machine or something?
Any help will be appreciated.
thanks.
Related
I am running Fusion 4.2.4 with external Zookeeper (3.5.6) and Solr (7.7.2). I have been running a local set of servers and am trying to move to AWS instances. All of the configuration from my local Zookeepers has been duplicated to the AWS instances so they should be functionally equivalent.
I am to the point where I want to shut down the old (local) Zookeeper instances and just use the ones running in AWS. I have changed the configuration for Solr and Fusion (fusion.properties) so that they only use the AWS instances.
The problem I have is that Fusion's solr cluster (System->Solr Clusters) associated with all of my collections is still set to the old Zookeepers :9983,:9983,:9983 so if I turn off all of the old instances of Zookeeper my queries through Fusion's Query API no longer work. When I try to change the "Connect String" for that cluster it fails because the cluster is currently in use by collections. I am able to create a new cluster but there is no way that I can see to associate the new cluster with any of my collections. In a test environment set up similar to production, I have changed the searchClusterId for a specific collection using Fusion's Collections API however after doing so the queries still fail when I turn off all of the "old" Zookeeper instances. It seems like this is the way to go so I'm surprised that it doesn't seem to work.
So far, Lucidworks's support has not been able to provide a solution - I am open to suggestions.
This is what I came up with to solve this problem.
I created a test environment with an AWS Fusion UI/API/etc., local Solr, AWS Solr, local ZK, and AWS ZK.
1. Configure Fusion and Solr to only have the AWS ZK configured
2. Configure the two ZKs to be an ensemble
3. Create a new Solr Cluster in Fusion containing only the AWS ZK
4. For each collection in Solr
a. GET the json from <fusion_url>:8764/api/collections/<collection>
b. Edit the json to change “searchClusterId” to the new cluster defined in Fusion
c. PUT the new json to <fusion_url>:8764/api/collections/<collection>
After doing all of this, I was able to change the “default” Solr cluster in the Fusion Admin UI to confirm that no collections were using it (I wasn’t sure if anything would use the ‘default’ cluster so I thought it would be wise to not take the chance).
I was able to then stop the local ZK, put the AWS ZK in standalone mode, and restart Fusion. Everything seems to have started without issues.
I am not sure that this is the best way to do it, but it solved the problem as far as I could determine.
I have been trying to implement a SolrCloud, and everything works fine until I try to create a collection with 6 shards. My setup is as follows:
5 virtual servers, all running Ubuntu 14.04, hosted by a single company across different data centers
3 servers running ZooKeeper 3.4.6 for the ensemble
2 servers, each running Solr 5.1.0 server (Jetty)
The Solr instances each have a 2TB, ext4 secondary disk for the indexes, mounted at /solrData/Indexes. I set this value in solrconfig.xml via <dataDir>/solrData/Indexes</dataDir>, and uploaded it to the ZooKeeper ensemble. Note that these secondary disks are neither NAS nor NFS, which I know can cause problems. The solr user owns /solrData.
All the intra-server communication is via private IP, since all are hosted by the same company. I'm using iptables for firewall, and the ports are open and all the servers are communicating successfully. Config upload to ZooKeeper is successful, and I can see via the Solr admin interface that both nodes are available.
The trouble starts when I try to create a collection using the following command:
http://xxx.xxx.xxx.xxx:8983/solr/admin/collections?action=CREATE&name=coll1&maxShardsPerNode=6&router.name=implicit&shards=shard1,shard2,shard3,shard4,shard5,shard6&router.field=shard&async=4444
Via the Solr UI logging, I see that multiple index creation commands are issued simultaneously, like so:
6/25/2015, 7:55:45 AM WARN SolrCore [coll1_shard2_replica1] Solr index directory '/solrData/Indexes/index' doesn't exist. Creating new index...
6/25/2015, 7:55:45 AM WARN SolrCore [coll1_shard1_replica2] Solr index directory '/solrData/Indexes/index' doesn't exist. Creating new index...
Ultimately the task gets reported as complete, but in the log, I have locking errors:
Error creating core [coll1_shard2_replica1]: Lock obtain timed out: SimpleFSLock#/solrData/Indexes/index/write.lock
SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
Error closing IndexWriter
If I look at the cloud graph, maybe a couple of the shards will have been created, others are closed or recovering, and if I restart Solr, none of the cores can fire up.
Now, I know what you're going to say: follow this SO post and change solrconfig.xml locking settings to this:
<unlockOnStartup>true</unlockOnStartup>
<lockType>simple</lockType>
I did that, and it had no impact whatsoever. Hence the question. I'm about to have to release a single Solr instance into production, which I hate to do. Does anybody know how to fix this?
Based on the log entry you supplied, it looks like Solr may be creating the data (index) directory for EACH shard in the same folder.
Solr index directory '/solrData/Indexes/index' doesn't exist. Creating new index...
This message was shown for two different collections and it references the same location. What I usually do, is change my Solr Home to a different directory, under which all collection "instance" stuff will be created. Then I manually edit the core.properties for each shard to specify the location of the index data.
I am playing with solrcloud 4.6.0 and have a question. I have 3 standalone zookeeper v3.4.5 servers and 2 solr instances. Zookeeper was freshly installed and empty.
I configured the first solr node with the new discovery format, prepared initial existing cores (with data) and started the first solr with the bootstrap_conf set to true.
By observing the solr log I saw the cores config was uploaded to zookeeper and can also confirm that by looking at zookeeper registry. Also the cores are visible in the solr web console, searchable. All in all working.
Now I wanted the second solr node to kick in. The second solr was only having solr.xml (exactly the same as the first solr node). My understanding was that starting the second solr node will read the cores info from zookeeper and after a while all cores will replicate to it.
Didn't happen.
No errors in the logs, second solr web console says no cores are available.
What did I miss?
Best regards
No errors in the log, huh? Bad eye I have.
It turns out that the initial configuration which was uploaded to zookeeper wasn't expending the variables in solr.xml, so the clusterstate.json had wrong ports for my Jetty solr installations - the default port 8983 was taken for replicas, but they were running on different ports.
I just modified the default value on all replicas in solr.xml file prior to booting the first solr instance with bootstrap_conf and now the cores are immediately visible. Also I deleted all zookeeper info before.
But still replicas have to be created on each node via web gui or by rest api.
I just started using apache solr with it's data import functionality on my project
by following steps in http://tudip.blogspot.in/2013/02/install-apache-solr-on-ubuntu.html
but now I have to make two different instances of my project on same server with different databases but with same configuration of solr for both projects. How can I do that?
Please help me if anyone can?
Probably the closest you can get is having two different Solr cores. They will run under the same server but will have different configuration (which you can copy paste).
When you say "different databases", do you mean you want to copy from several databases into one join collection/core? If so, you just define multiple entities and possibly multiple datasources in your DataImportHandler config and run either all of them or individually. See the other question for some tips.
If, on the other hand, you mean different Solr cores/collections, then you just want to run Solr with multiple cores. It is very easy, you just need solr.xml file above your collection level as described on the Solr wiki. Once you get the basics, you may want to look at sharing instance directories and having separate data directories to avoid changing the same config twice (instanceDir vs. dataDir settings on each core).
I am learning how to use Solrcloud's new features, and I can successfully set up an ensemble of Zookeepers, and a set of Solr instances for a sharded index. I wanted to investigate how failures affected my setup. Mostly, it worked as expected except for one case.
I used two machines, and started 3 Zookeepers on each (6 total). I started s Solr instance one machine (bosmac01), asking for 2 shards, and started a second instance that machine. I then started two more Solr instances on a second machine (qasolrmaster). The Solr admin showed the configuration I expected, and indexing/querying worked:
Shard1: qasolrmaster:8900 and bosmac01:8983
Shard2: qasolrmaster:8910 and bosmac01:8920
I wanted to test what would happen if one machine crashed, so I shutdown qasolrmaster. I expected that since there would be 3 Zookeepers still running, and since there would still be a Solr instance connected to each shard, that everything would still work. Instead, the two remaining Solr instances (on bosmac01) kept trying to reconnect to the missing Zookeepers. The Admin would not display the cloud image, and I could not add docs or query. Same thing happens if I just stop all the Zookeepers on qasolrmaster but leave the machine running. Re-starting one of the missing Zookeepers returned things to normal.
Why did the test fail? 3 Zookeepers plus a Solr for each shard should allow things to keep working, yes?
Zk requires a majority of its nodes stay up. If you put 3 on one machine and 3 on another, then kill 3, you do not have a majority.