SOLR Zookeeper CLI failing - IOException Packet len is out of range - solr

I wanted to run Security Authentication and Authorization in SOLR 5.3.1 and placed the security.json file in solr root folder but on running the CLI as below :
/cygdrive/c/xampp/htdocs/solr$ server/scripts/cloud-scripts/zkcli.bat -zkhost localhost:8983 -cmd putfile /security.json security.json
I am getting the below issue :
WARN - 2015-10-29 15:39:01.656; org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server frontend.dev/127.0.0.1:8983, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Packet len1213486160 is out of range!
at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
I run through the below article :
Zookeeper CLI failing - IOException Packet <len12343123123> is out of range
I followed the instruction to increase the buffer size.I increased it as below for zkcli.bat . The command is as below :
zkcli.bat
#echo off
REM You can override pass the following parameters to this script:
REM
set JVM=java
REM Find location of this script
set SDIR=%~dp0
if "%SDIR:~-1%"=="\" set SDIR=%SDIR:~0,-1%
"%JVM%" -Dlog4j.configuration="file:%SDIR%\log4j.properties" -Djute.maxbuffer="20213486160" -classpath "%SDIR%\..\..\solr-webapp\webapp\WEB-INF\lib\*;%SDIR%\..\..\lib\ext\*" org.apache.solr.cloud.ZkCLI %*
even I set the maxClientCnxns,minSessionTimeout and maxSessionTimeout in zoo.cfg under server\solr :
zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
maxClientCnxns=500
minSessionTimeout=100000
maxSessionTimeout=200000
security.json file is as follows :
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{"solr":"admin"},
"permissions":[{"name":"security-edit",
"role":"admin"}]
}}
I googled but aint get any solution.Please kindly assist what I need to do to make the security authentication and authorization work in SOLR.

I had the same problem, but in my case was that solr 6 was running on normal mode and not in cluster mode. To run solr in cluster mode you need the -c flag.
Then the zookeeper was running on port 911.
./solr status
And see if you have a zookeeper running and in which port

Related

Solr / Zookeeper : "An exception was thrown while closing send thread"

I am trying Solr for the first time on RHEL 8 with Openjdk version "17.0.2".
I am following the tutorial https://solr.apache.org/guide/8_11/solr-tutorial.html. I get the warning:
WARN - 2022-04-20 12:07:20.762; org.apache.zookeeper.ClientCnxn; An exception was thrown while closing send thread for session 0x10003e1057e0003. => EndOfStreamException: Unable to read additional data from server sessionid 0x10003e1057e0003, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x10003e1057e0003, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) ~[zookeeper-3.6.2.jar:3.6.2]
This should be a straight forward tutorial. Do you know what I am missing?
Here is tutorial from the start:
[solr#abc294837 ~]$ ./bin/solr start -e cloud
Welcome to the SolrCloud example!
This interactive session will help you launch a SolrCloud cluster on your local workstation.
To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2]:
Ok, let's start up 2 Solr nodes for your example SolrCloud cluster.
Please enter the port for node1 [8983]:
Please enter the port for node2 [7574]:
Solr home directory /opt/solr/example/cloud/node1/solr already exists.
/opt/solr/example/cloud/node2 already exists.
Starting up Solr on port 8983 using command:
"/opt/solr/bin/solr" start -cloud -p 8983 -s "/opt/solr/example/cloud/node1/solr"
Waiting up to 180 seconds to see Solr running on port 8983 [\]
Started Solr server on port 8983 (pid=50226). Happy searching!
Starting up Solr on port 7574 using command:
"/opt/solr/bin/solr" start -cloud -p 7574 -s "/opt/solr/example/cloud/node2/solr" -z localhost:2181
Waiting up to 180 seconds to see Solr running on port 7574 [-]
Started Solr server on port 7574 (pid=50417). Happy searching!
INFO - 2022-04-20 12:07:20.502; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
INFO - 2022-04-20 12:07:20.553; org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
INFO - 2022-04-20 12:07:20.556; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
INFO - 2022-04-20 12:07:20.631; org.apache.solr.common.cloud.ZkStateReader; Updated live nodes from ZooKeeper... (0) -> (2)
INFO - 2022-04-20 12:07:20.737; org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at localhost:2181 ready
WARN - 2022-04-20 12:07:20.762; org.apache.zookeeper.ClientCnxn; An exception was thrown while closing send thread for session 0x10003e1057e0003. => EndOfStreamException: Unable to read additional data from server sessionid 0x10003e1057e0003, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x10003e1057e0003, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) ~[zookeeper-3.6.2.jar:3.6.2]
Now let's create a new collection for indexing documents in your 2-node cluster.
Please provide a name for your new collection: [gettingstarted]
ยดยดยด
You are not missing anything, this is Zookeeper falsely warning about a socket connection being closed.
[EDIT] : This has been fixed in Solr versions 8.11.2, and 9.0.0 (Zookeeper versions 3.6.4, 3.7.1, 3.8.1, 3.9.0).
We can see in this commit that the exception is caught and expected (comment says closing so this is expected), yet it is now reported as a warning and a stack trace is logged, although this is not an error per se. So you can consider this message a debug message (as it was before that commit).
See for reference this issue, caused by this issue, and this pull request for the fix.
We can still make Zookeeper quiet from Solr/log4j config, by changing the level of its logger from "warn" to "error" :
solr/solr/server/resources/log4j2-console.xml
<AsyncLogger name="org.apache.zookeeper" level="ERROR"/>

Indexing problem with SOLR (MultiMaxScoreQParserPlugin)

I'm trying to integrate SOLR with Hybris but both of them are running on Kubernetes as a diffferent pod.
If I'm trying indexing SOLR on Hybris, it throws the error below;
ERROR [BackofficeLO-47] (000001JT) [SolrStandaloneSearchProvider] Error from server at http://10.10.100.181:34324/solr: Error CREATEing SolrCore 'master_backoffice_backoffice_product_flip': Unable to create core [master_backoffice_backoffice_product_flip] Caused by: de.hybris.platform.solr.search.MultiMaxScoreQParserPlugin
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://10.10.100.181:34324/solr: Error CREATEing SolrCore 'master_backoffice_backoffice_product_flip': Unable to create core [master_backoffice_backoffice_product_flip] Caused by: de.hybris.platform.solr.search.MultiMaxScoreQParserPlugin
I guess somethings wrong with SOLR "deafult" indexing directory.
Solr is running as process like below inside the pod;
solr#solr-fsd33wdf-qteg:/opt/solr-8.5.2$ ps -ef | grep solr
solr 10 1 0 Jun23 ? 00:15:55 /usr/local/openjdk-11/bin/java -server -Xms512m -Xmx512m -XX:+UseG1GC -XX:+PerfDisableSharedMem -XX:+ParallelRefProcEnabled -XX:MaxGCPauseMillis=250 -XX:+UseLargePages -XX:+AlwaysPreTouch -Xlog:gc*:file=/var/solr/logs/solr_gc.log:time,uptime:filecount=9,filesize=20M -Dsolr.jetty.inetaccess.includes= -Dsolr.jetty.inetaccess.excludes= -Dsolr.log.dir=/var/solr/logs -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/opt/solr/server -Dsolr.solr.home=/var/solr/data -Dsolr.data.home= -Dsolr.install.dir=/opt/solr -Dsolr.default.confdir=/opt/solr/server/solr/configsets/_default/conf -Dlog4j.configurationFile=/var/solr/log4j2.xml -Xss256k -Dsolr.jetty.https.port=8983 -jar start.jar --module=http
So default confdir is /opt/solr/server/solr/configsets/_default/conf
If I'm check SOLR_HOME variable, it's different directory;
solr#solr-f575dcfdf-qtnpg:/opt/solr-8.5.2$ echo $SOLR_HOME
/var/solr/data
So, how can I change confdir to /var/solr/data ? I guess this is the problem here?
Thanks!
This page provides you with information on how to use the standalone setup. The ant configureSolrServer takes an argument of the path to the original Solr binary that you should download from here. This overwrites the files in the directory with the SAP Commerce specific setup. The MultiMaxScoreQParserPlugin is part of solr-hybris-components-<version_of_solr>.jar file, where the <version_of_solr> corresponds to the solr version your SAP Commerce is running on. Note that SAP Commerce also supports multiple Solr versions and it depends on what configuration you have.
You may then extend the default Solr docker image as provided here to have your setup running.

Download Solr Collection config from specific directory on ZK

We have Solr collections' config details in a specific directory on Zookeeper. Below is the directory structure on zookeeper;
|--app_dir
|--dev
|--configs
|--collection_name
|--conf
I am trying to download the configuration of collections using downconfig utility. Following is the command I am using;
bin/solr zk downconfig -n conf_name -d /home/ec2-user/solr-8.1.0/server/solr/configsets -z zk-dev-1:2181,zk-dev-2:2181,zk-dev-3:2181
The command fails to execute with below error,
ERROR: Error downloading files from zookeeper path /configs/test_vector to /home/ec2-user/solr-8.1.0/server/solr/configsets/conf
ERROR - 2020-09-30 15:11:23.778; org.apache.solr.util.SolrCLI; Could not complete downconfig operation for reason: Error downloading files from zookeeper path /configs/test_vector to /home/ec2-user/solr-8.1.0/server/solr/configsets/conf
I suspect the path for config is incorrect as the actual path is app_name/dev/configs/collection_name while the error suggests the path is only configs/collection_name but I am not sure if that's the error.
Appreciate the help!
If you're not using the default /solr path inside ZK, you should be able to change it using
-zkhost 127.0.0.1:2181/app_dir/dev
This is also required for any configuration of a SolrCloud compatible (i.e. Zookeeper-aware) client or other operations that interact with Zookeeper.

Solr: find ULR and port

I have solr-4.10.3 installed in my Ubuntu server, I have forgot the URL and port.
Where can I find this infos in my Solr config files ?
Thanks for help.
The admin UI should be accessible at
http://localhost:8983/solr
Default port: 8983
To change a port
java -Djetty.port=9999 -jar start.jar
For solr 5.4.1 for example
solr-5.4.1\bin\solr start -p XXXX
Please visit https://cwiki.apache.org/confluence/display/solr/Taking+Solr+to+Production for more information.
Go inside your Solr directory. Navigate to example folder,As per your Solr insatance it will be like solr-4.10.3/example/
then start your Solr instance from Ubuntu terminal using command
java -jar start.jar
Open browser and hit URL
http://localhost:8983/solr

Reloading SolrCloud configuration (stored on Zookeeper) - schema.xml

I have setup a SolrCloud replication using standalone zookeeper. But now I wish to make some changes to my Schema.xml and reload the core. The problem is that when I run a single server Solr (no solrcloud) the new schema is loaded, but I do not know how to reload schema on all the replication server. I tried reloading the schema on one of the server with no desired impact. Is there a way in which I can reload my schema.xml in Solr in distributed replication setup which uses zookeeper.
Just found the solution we need to push the changed configuration to zookeeper ensemble.
Just use
sh zkcli.sh -cmd upconfig -zkhost 127.0.0.1:2181 -collection collection1 -confname myconf -solrhome ../solr -confdir ../solr/collection1/conf
zkcli.sh is present under example/cloud-scripts
The answer marked as correct is wrong. You have to use Solr Collection API
Once you have uploaded the new collection (index) configuration with the Solr zkcli.sh utility the configuration will not be reloaded automatically.
Solr Collection API are indicated for SolrCloud and the configuration reload will be spread in the whole cluster. As far as I know Solr Collection API are available at least from Solr 4.8.
The procedure is slightly different and with these API you can reload the configuration on the entire Cluster with only one API call.
Just upload your updated configuration with the Solr zkcli.sh utility. Pay attention to do not confuse Solr zkcli.sh with Zookeeper zkCli.sh they have quite the same name but completely different purpose.
So as said use Solr zkcli.sh (At time of writing is in the directory server/scripts/cloud-scripts):
./zkcli.sh -cmd upconfig -zkhost 127.0.0.1:2181 -collection collection1 -confname myconf -confdir path/to/solr/collection1/conf
Then you can reload the configuration of collection1 with:
http://server1:8983/solr/admin/collections?action=RELOAD&name=collection1
The entire cluster will be updated.
This worked for me :
bin/solr zk -upconfig -n collectionName -d pathto/Conf_directory -z localhost:2181/solr
Below is the Command for Windows,
IT will be almost same in Unix we just need to change the path of Solr lib and class-path separator ; & : Because its java command so should run in Unix also.
java -Dlog4j.configuration="file:E:/solr-5.5.1/server/scripts/cloud-scripts/log4j.properties" -classpath .;E:/solr-5.5.1/server/solr-webapp/webapp/WEB-INF/lib/*;E:/solr-5.5.1/server/lib/ext/* org.apache.solr.cloud.ZkCLI -cmd upconfig -zkhost 192.168.42.13:2787 -confdir E:/New_Solor_Conf -confname Solor_conf
Brief details about command as follows:
Configuration of log4j for logging.
-Dlog4j.configuration="file:E:/solr-5.5.1/server/scripts/cloud-scripts/log4j.properties
Class path to run "org.apache.solr.cloud.ZkCLI". class.
make sure UNIX and Windows will have different : (Unix seperator) ;(Windows Separator)
-classpath .;E:/solr-5.5.1/server/solr-webapp/webapp/WEB-INF/lib/;E:/solr-5.5.1/server/lib/ext/
-zkhost 192.168.42.13:2787 (Remote Host and port where Solr Zookeeper is running)
-confdir E:/New_Solor_Conf (Local directory what we need to upload.)
-confname Solor_conf Remote instance name.
If you will not use correct class path you will get error like :
Error: Could not find or load main class org.apache.solr.cloud.ZkCLI
or
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFacto
ry
at org.apache.solr.common.cloud.SolrZkClient.<clinit>(SolrZkClient.java:
71)
at org.apache.solr.cloud.ZkCLI.main(ZkCLI.java:183)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
I am able to upload my local configuration changes without physically login to remote Solr box. Hope it will work for other also.

Resources