I have a SolrCloud instance running with a single core / collection.
I am attempting to download the configuration for this collection with the following command:
/opt/solr-5.3.0/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:9983 -cmd downconfig -confdir /opt/solr/collection1 -confname *
However, I get the following error:
Exception in thread "main" java.io.IOException: Error downloading files from zookeeper path /configs/bin to /opt/solr/collection1
at org.apache.solr.common.cloud.ZkConfigManager.downloadFromZK(ZkConfigManager.java:107)
at org.apache.solr.common.cloud.ZkConfigManager.downloadConfigDir(ZkConfigManager.java:131)
at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:230)
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /configs/bin
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:328)
at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:325)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:325)
at org.apache.solr.common.cloud.ZkConfigManager.downloadFromZK(ZkConfigManager.java:92)
I do not know the confname so I am provding * as its value. Is that the cause of the issue?
All that I wish to know is how to download the configuration for the existing core / collection (which I then intend to upload to my own local installation)
Found out the cause of the issue. It was the value passed to confname option.
The confname option is a mandatory option when attempting to download the configurations of an existing core / collection.
Turns out that when a configuration is uploaded to zookeeper, you don't have to specify the confname option - in such a case, zookeeper uses the collection name itself as the configuration name.
My collection was named Collection1 and thus, by providing that I managed to successfully download the configuration.
The final command was:
/opt/solr-5.3.0/server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:9983
-cmd downconfig -confdir /opt/solr/collection1 -confname Collection1
Related
We have Solr collections' config details in a specific directory on Zookeeper. Below is the directory structure on zookeeper;
|--app_dir
|--dev
|--configs
|--collection_name
|--conf
I am trying to download the configuration of collections using downconfig utility. Following is the command I am using;
bin/solr zk downconfig -n conf_name -d /home/ec2-user/solr-8.1.0/server/solr/configsets -z zk-dev-1:2181,zk-dev-2:2181,zk-dev-3:2181
The command fails to execute with below error,
ERROR: Error downloading files from zookeeper path /configs/test_vector to /home/ec2-user/solr-8.1.0/server/solr/configsets/conf
ERROR - 2020-09-30 15:11:23.778; org.apache.solr.util.SolrCLI; Could not complete downconfig operation for reason: Error downloading files from zookeeper path /configs/test_vector to /home/ec2-user/solr-8.1.0/server/solr/configsets/conf
I suspect the path for config is incorrect as the actual path is app_name/dev/configs/collection_name while the error suggests the path is only configs/collection_name but I am not sure if that's the error.
Appreciate the help!
If you're not using the default /solr path inside ZK, you should be able to change it using
-zkhost 127.0.0.1:2181/app_dir/dev
This is also required for any configuration of a SolrCloud compatible (i.e. Zookeeper-aware) client or other operations that interact with Zookeeper.
I added a misconfigured dynamic field to a Solr Core. Since, I cannot update anything to fix this error because Solr fails to load it.
The erroneous query:
http://solr.dev.fr:8983/solr/zCollection/schema
{
"add-dynamic-field":{
"name":"*_alz*",
"type":"customFieldType",
"stored":true,
"indexed":true
}
The Exception :
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Could not load conf for core zCollection_shard1_replica1: Can't load schema managed-schema: Dynamic field name '*_alz*' should have either a leading or a trailing asterisk, and no others.
The problem:
I can't find this dynamic field anywhere because I'm using data_driven_schema_configs
I can't use schema API to delete it; I get 404 Not Found in response.
The Question:
Where can I found this element and delete it?
PS: I did a
grep -rnw '/opt/lucidworks-hdpsearch/' -e '_alz'
But nothing comes out with me.
Update 1 :
I found the field in Zookeeper files using:
./zkcli.sh -zkhost hmaster.dev.fr:2181 -cmd list
I downloaded the file
./zkcli.sh -zkhost hmaster.dev.fr:2181 -cmd get /configs/zCollection/managed-schema
Fixed the erroneous fields and uploaded it to ZK again:
./zkcli.sh -zkhost hmaster.dev.fr:2181 -cmd putfile /configs/zCollection/managed-schema managed-schema
And it works finally!!
I have a collection in solrcloud which had been created using zookeeper managed configuration and I want all collection configuration files which were used to create the collection. Here are the options I found:
Copy manually all files from Solrcloud UI.
solrUI->cloud->tree->/collections/<collection-name>
Download files from zookeeper
/opt/solr/server/scripts/cloud-scripts/zkcli.sh -cmd downconfig -zkhost <zk hosts>/usecasedir -confname <configuration name> -confdir <dir to download>
2nd option would save my lot of time but the problem here my zookeeper has huge list of configurations and I am not sure which configuration directory was used to create collection.
Is there any way to figure out which collection configuration was used to create collection?
the info of what config was used to create a collection is stored in zk itself. Some bash scripting (using the great jq utility) is enough to do what you need:
find what config was used for the given XXX collection:
CONFIGNAME=$(curl -L -s "http://localhost:8983/solr/admin/zookeeper?detail=true&path=/collections/XXX" | jq '.znode.data' | cut -d ":" -f2 | tr -d '}"\\')
now download the config:
/opt/solr/bin/solr zk downconfig -n $CONFIGNAME -d config$CONFIGNAME -z localhost:2181
Sets of configuration are usually found in a directory called /configs. If the zookeeper is dedicated to solr this is usually at the top level, if it's used by multiple applications it's common to "zk chroot" the configs to a sub directory.
Once you find the right location in zookeeper, one directory in the configs directory should match the name shown as "config-name" in the admin UI under Collections > name_of_your_collection
If your project uses gradle, the up/down load of configs from the project (where you might want to check these things into version control) can be smoothed somewhat by a plugin (disclaimer: I wrote this plugin)
https://plugins.gradle.org/plugin/com.needhamsoftware.solr-gradle
There's an additional complication to be aware of however, if the collection is using managed schema, the acutal schema in use will not be in schema.xml, but in a file called "managed-schema"
Fields may have been added via the Schema Rest API, so "files used to create the collection" is a bit fuzzy in that respect, but the managed_schema can be renamed to schema.xml and the solr config modified to take things out of managed mode if you want.
I am attempting to create a core in SolrCloud using the following command:
sudo /opt/solr/bin/solr create -c Collection1 -d /home/callmekatootie/Desktop/conf
However, I get the following error:
Connecting to ZooKeeper at localhost:9983 ...
Uploading /home/callmekatootie/Desktop/vm/configs for config Collection1
to ZooKeeper at localhost:9983
Creating new collection 'Collection1' using command:
http://localhost:8983/solr/admin/collections?action=CREATE&name=Collection1
&numShards=1&replicationFactor=1&maxShardsPerNode=1
&collection.configName=Collection1
ERROR: Failed to create collection 'Collection1' due to: org.apache.solr.client.
solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at
http://127.0.1.1:8983/solr: Error CREATEing SolrCore 'Collection1_shard1_replica1':
Unable to create core [Collection1_shard1_replica1]
Caused by: solr.SynonymExpandingExtendedDismaxQParserPlugin
What is the SynonymExpandingExtendedDismaxQParserPlugin and why am I getting the above error
Found the cause of the issue.
Turns out that SynonymExpandingExtendedDismaxQParserPlugin is an actual plugin that was being used in the SolrConfig.xml present in the configuration files that I was uploading.
This plugin was available through https://github.com/healthonnet/hon-lucene-synonyms which I had to copy to /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/ folder - I had to paste that jar file in this folder. I restarted Solr and then managed to execute the mentioned command successfully.
I have setup a SolrCloud replication using standalone zookeeper. But now I wish to make some changes to my Schema.xml and reload the core. The problem is that when I run a single server Solr (no solrcloud) the new schema is loaded, but I do not know how to reload schema on all the replication server. I tried reloading the schema on one of the server with no desired impact. Is there a way in which I can reload my schema.xml in Solr in distributed replication setup which uses zookeeper.
Just found the solution we need to push the changed configuration to zookeeper ensemble.
Just use
sh zkcli.sh -cmd upconfig -zkhost 127.0.0.1:2181 -collection collection1 -confname myconf -solrhome ../solr -confdir ../solr/collection1/conf
zkcli.sh is present under example/cloud-scripts
The answer marked as correct is wrong. You have to use Solr Collection API
Once you have uploaded the new collection (index) configuration with the Solr zkcli.sh utility the configuration will not be reloaded automatically.
Solr Collection API are indicated for SolrCloud and the configuration reload will be spread in the whole cluster. As far as I know Solr Collection API are available at least from Solr 4.8.
The procedure is slightly different and with these API you can reload the configuration on the entire Cluster with only one API call.
Just upload your updated configuration with the Solr zkcli.sh utility. Pay attention to do not confuse Solr zkcli.sh with Zookeeper zkCli.sh they have quite the same name but completely different purpose.
So as said use Solr zkcli.sh (At time of writing is in the directory server/scripts/cloud-scripts):
./zkcli.sh -cmd upconfig -zkhost 127.0.0.1:2181 -collection collection1 -confname myconf -confdir path/to/solr/collection1/conf
Then you can reload the configuration of collection1 with:
http://server1:8983/solr/admin/collections?action=RELOAD&name=collection1
The entire cluster will be updated.
This worked for me :
bin/solr zk -upconfig -n collectionName -d pathto/Conf_directory -z localhost:2181/solr
Below is the Command for Windows,
IT will be almost same in Unix we just need to change the path of Solr lib and class-path separator ; & : Because its java command so should run in Unix also.
java -Dlog4j.configuration="file:E:/solr-5.5.1/server/scripts/cloud-scripts/log4j.properties" -classpath .;E:/solr-5.5.1/server/solr-webapp/webapp/WEB-INF/lib/*;E:/solr-5.5.1/server/lib/ext/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 192.168.42.13:2787 -confdir E:/New_Solor_Conf -confname Solor_conf
Brief details about command as follows:
Configuration of log4j for logging.
-Dlog4j.configuration="file:E:/solr-5.5.1/server/scripts/cloud-scripts/log4j.properties
Class path to run "org.apache.solr.cloud.ZkCLI". class.
make sure UNIX and Windows will have different : (Unix seperator) ;(Windows Separator)
-classpath .;E:/solr-5.5.1/server/solr-webapp/webapp/WEB-INF/lib/;E:/solr-5.5.1/server/lib/ext/
-zkhost 192.168.42.13:2787 (Remote Host and port where Solr Zookeeper is running)
-confdir E:/New_Solor_Conf (Local directory what we need to upload.)
-confname Solor_conf Remote instance name.
If you will not use correct class path you will get error like :
Error: Could not find or load main class org.apache.solr.cloud.ZkCLI
or
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFacto
ry
at org.apache.solr.common.cloud.SolrZkClient.<clinit>(SolrZkClient.java:
71)
at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:183)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
I am able to upload my local configuration changes without physically login to remote Solr box. Hope it will work for other also.