When I running command bin/solr start -e cloud it is not asking me to collection name and other information like no of replicas and configuration settings.
I got following output
Welcome to the SolrCloud example!
This interactive session will help you launch a SolrCloud cluster on your local workstation.
To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2] 2
Ok, let's start up 2 Solr nodes for your example SolrCloud cluster.
Please enter the port for node1 [8983]
8983
Please enter the port for node2 [7574]
7574
Creating Solr home directory /home/webuser/abhijeet/solr-5.0.0/example/cloud/node1/solr
Cloning Solr home directory /home/webuser/abhijeet/solr-5.0.0/example/cloud/node1 into /home/webuser/abhijeet/solr-5.0.0/example/cloud/node2
Starting up SolrCloud node1 on port 8983 using command:
solr start -cloud -s example/cloud/node1/solr -p 8983
Started Solr server on port 8983 (pid=23735). Happy searching!
I tried it ...here it goes...only difference is I tried on Ubantu
abhijit#abhijit:~/Downloads/solr-5.0.0$ bin/solr start -e cloud
Welcome to the SolrCloud example!
This interactive session will help you launch a SolrCloud cluster on your local workstation.
To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2] 2
Ok, let's start up 2 Solr nodes for your example SolrCloud cluster.
Please enter the port for node1 [8983] 8983
Oops! Looks like port 8983 is already being used by another process. Please choose a different port. 8985
8985
Please enter the port for node2 [7574] 7576
7576
Starting up SolrCloud node1 on port 8985 using command:
solr start -cloud -s example/cloud/node1/solr -p 8985
Waiting to see Solr listening on port 8985 [|]
Started Solr server on port 8985 (pid=7092). Happy searching!
Starting node2 on port 7576 using command:
solr start -cloud -s example/cloud/node2/solr -p 7576 -z localhost:9985
Waiting to see Solr listening on port 7576 [\]
Started Solr server on port 7576 (pid=7461). Happy searching!
Now let's create a new collection for indexing documents in your 2-node cluster.
Please provide a name for your new collection: [gettingstarted] testCore
testCore
How many shards would you like to split testCore into? [2] 2
2
How many replicas per shard would you like to create? [2] 2
2
Please choose a configuration for the testCore collection, available options are:
basic_configs, data_driven_schema_configs, or sample_techproducts_configs [data_driven_schema_configs] data_driven_schema_configs
Connecting to ZooKeeper at localhost:9985
Uploading /home/abhijit/Downloads/solr-5.0.0/server/solr/configsets/data_driven_schema_configs/conf for config testCore to ZooKeeper at localhost:9985
Creating new collection 'testCore' using command:
http://127.0.1.1:7576/solr/admin/collections?action=CREATE&name=testCore&numShards=2&replicationFactor=2&maxShardsPerNode=2&collection.configName=testCore
{
"responseHeader":{
"status":0,
"QTime":7468},
"success":{"":{
"responseHeader":{
"status":0,
"QTime":7057},
"core":"testCore_shard1_replica2"}}}
SolrCloud example running, please visit http://localhost:8985/solr
Related
I have to do the following:
I have to deploy Solr on 2 servers/nodes.
Deploy Zookeeper on another server.
Upload a custom config to Zookeeper
Create a custom collection with 2 shards and 2 replicas
Version of Solr 7.4.0 & Zookeeper: 3.4.12
I have done the following:
Set Up Zookeeper:
Created a Zookeeper data folder & made a zoo.conf & put the dataDir there.
Started zookeeper using ./zkServer.sh start
Set up Solr:
Started Solr using:
./solr start -cloud -s /home/demo/LocalFolder/Downloads/SolrHome -p 8987 -z localhost:2181
Trying to upload config in Zookeeper using:
./solr create -c mycollection -d /media/sf_VM/Dump/conf
It is giving me an exception:
Caused by: javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
... 17 more
</pre>
I have searched many pages & seen Solr tutorials but there they have used the default examples. I just dont have any step by step idea to
How to Upload a config in Zookeeper?
Then what I need to do to create a collection pointing to that config. I want that collection to have 2 shards & 2 replicas.
Where will be the solr.xml. If it should be in Zookeeper how do I upload it there
How do I see in Zookeeper that the config has been uploaded?
I know this question might be a duplicate. I have read several of posts but not able to come up a solution. Please help.
Well I figured out how to setup. Please note that I have read these steps from different site & gathered here step by step:
1. Set Up ZooKeeper:
This is used to keep the collection specific configuration files to store in a central space & map it with a name. Later we can use this name to create collection which would point to this config. This config is the same as you find in the below folder:
solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf
1.1 Download latest Zookeeper. I have used 3.4.12
1.2 Unpack downloaded archive and copy conf/zoo_sample.cfg to conf/zoo.cfg
1.3 Modify zoo.cfg:
1.3.1 Change dataDir to directory where you want to hold all cluster configuration data.
dataDir=/var/zookeeper/data
1.3.2 Add information about all Zookeeper servers: I have used only 1 ZooKeeper server so this is not required. If you want to add more servers then please go through the below links:
DZone Tutorial
Apache Tutorial
1.3.3 Start ZooKeeper using (after going to zookeeper-3.4.12/):
./bin/zkServer.sh start-foreground conf/zoo.cfg
OR
./bin/zkServer.sh start conf/zoo.cfg
Note: You can stop the ZooKeeper using the below command:
bin/zkServer.sh stop
1.3.4 ZooKeeper Status:
Hit the below:
bin/zkServer.sh status
Or do telnet localhost 2181 and hit stats when connected.
2. Set Up Solr
2.1 Download the Solr
2.2 Extract install_solr_service.sh from the .tar file. Solr includes a service installation script (bin/install_solr_service.sh) to help you install Solr as a service on Linux. For more info click here.
tar -xzf solr-7.4.0.tgz solr-7.4.0/bin/install_solr_service.sh --strip-components=2
2.3 Install Solr as a service using the above script:
sudo bash ./install_solr_service.sh solr-7.4.0.tgz
This will also extracting solr-7.4.0.tgz to /opt/solr
2.4 Go to /opt/solr and do the following:
mkdir solr/server/solr2
mkdir solr/server/solr3
mkdir solr/server/solr4
cp solr/server/solr/solr.xml solr/server/solr2
cp solr/server/solr/solr.xml solr/server/solr3
cp solr/server/solr/solr.xml solr/server/solr4
2.5 Change the jetty port in solr.xml. Do this for all the 3 solr.xml mentioned in the above step:
vi solr/server/solr2/solr.xml
Search for the port 8983 & change it 8984 (for solr2), change it 8985 (for solr3), change it 8986 (for solr4)
2.6 Stop the Solr running at 8983
root#dev-base:/opt/solr/bin# ./solr stop -p 8983
2.7 Start all the solr instances:
root#dev-base:/opt/solr# bin/solr start -c -s server/solr -p 8983 -z localhost:2181 -noprompt -force
root#dev-base:/opt/solr# bin/solr start -c -s server/solr2 -p 8984 -z localhost:2181 -noprompt -force
root#dev-base:/opt/solr# bin/solr start -c -s server/solr3 -p 8985 -z localhost:2181 -noprompt -force
root#dev-base:/opt/solr# bin/solr start -c -s server/solr4 -p 8986 -z localhost:2181 -noprompt -force
Note: Running solr as a root is not recommended for security reason.
2.8 See Solr Status:
root#dev-base:/opt/solr# bin/solr status
3 Make a Custom Config
3.1 Copy the conf directory from solr-7.4.0\server\solr\configsets\sample_techproducts_configs\conf to another location (In my case it is /media/sf_VM/Dump/new/conf ).
3.2 Change the managed-schema file inside conf to specify the fields you are using.
4 Upload the config to ZooKeeper:
root#dev-base:/opt/solr# bin/solr zk -z localhost:2181 upconfig -d /media/sf_VM/Dump/new/conf -n myConf6
The name of the config I have uploaded is myConf6
5 Create a Solr Collection using this custom config
root#dev-base:/opt/solr# bin/solr create -c myNewCollection -n myConf6 -shards 2 -replicationFactor 2 -force
Hit Solr Admin URL
6 Index Data Using POST API using Json
URL: http://localhost:8983/solr/myNewCollection/update
Method: POST
Body:
[{
"_id": "99999999999999999999",
"author": [
"New Inserted 9000"
],
"authorLastName": [
"New Inserted 9000"
],
"impn": "New Inserted 9000",
"isbn10": "9999999999",
"isbn13": "9999999999999",
"title": "New Inserted 9000",
"publisher": "New Inserted 9000",
"sales_a": 5.0,
"sales_t": 5.0,
"haveImage": 1,
"pages": "76",
"image": "http://ip.ip.com/is/image/",
"format": "Paper",
"mtc_id": "99999999999",
"subjects" : [
"9000"
]
"rating": 0,
"description_long": "Snahashis call me in your marriage."
}
Delete a configuration in Zookeeper:
If you want to delete an old/wrong configuration already uploaded to ZooKeeper run the below command:
./server/scripts/cloud-scripts/zkcli.sh -cmd clear -z "<ZK_HOST>:<ZK_PORT>" /configs/AAA
The path of the configuration is /configs/< name of the configset >
To delete a specific file:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd clear /configs/<MY_COLLECTION>/solrconfig.xml
Upload an updated file:
zkcli.sh --zkhost <ZK_HOST>:<ZK_PORT> -cmd putfile /configs/<MY_COLLECTION>/solrconfig.xml /<MY_UPDATED_FILE_LOCAL_FOLDER>/solrconfig.xml
Then we need to restart the solr nodes.
To Delete a Collection Via API:
First delete alias created on this collection (if any)
http://localhost:8983/solr/admin/collections?action=DELETEALIAS&name=aliasName
Delete Collection:
http://localhost:8983/solr/admin/collections?action=DELETE&name=collectionName
How to upgrade Apache solr from existing version 4.10.4 to 6.6.0 (A latest one)
On linux centos 7 operating system.
stop existing running solr
sudo service solr stop
confirm its stopped or not
sudo service solr status
You can take backup of existing solr data from
/var/solr/data/< core_name >, But reindexing solr data after upgrading
solr version will give best performance without any unknown issues.
now remove existing solr from centos
sudo rm -r /var/solr
remove solr from following location if its there
sudo rm -r /opt/solr
sudo rm /etc/init.d/solr
In my case following path was not exist to delete, Delete if exists in your case
sudo rm -r /opt/solr/solr-4.10.4
remove existing solr user with home directory
sudo userdel -r solr
for ubuntu
sudo deluser --remove-home solr
remove existing group
sudo groupdel solr
for ubuntu
sudo deluser --group solr
restart your system
sudo reboot
Check supporting application and download solr 6.6
Check which version Java version is installed on your machine
java -version
Check available java versions, and update to latest version
yum list available java*
Install latest java version
sudo yum install <jdk version>
download and extract installation folder
mkdir /home/user/temp
cd temp
wget http://apache.org/dist/lucene/solr/6.6.0/solr-6.6.0.tgz
tar -zxvf solr-6.6.0.tgz
install solr
cd solr-6.6.0/
sudo bin/install_solr_service.sh /home/user/temp/solr-6.6.0.tgz
after successful installation you can see following output
id: solr: no such user Creating new user: solr
Extracting /home/user/temp/solr-6.6.0.tgz to /opt
Installing symlink /opt/solr -> /opt/solr-6.6.0 ...
Installing /etc/init.d/solr script ...
Installing /etc/default/solr.in.sh ...
Service solr installed. Customize Solr startup configuration in
/etc/default/solr.in.sh Waiting up to 180 seconds to see Solr running
on port 8983 [] Started Solr server on port 8983 (pid=4802). Happy
searching!
Found 1 Solr nodes:
Solr process 4802 running on port 8983 {
"solr_home":"/var/solr/data", "version":"6.6.0
5c7a7b65d2aa7ce5ec96458315c661a18b320241 - ishan - 2017-05-30
07:32:53", "startTime":"2017-06-26T07:02:51.523Z", "uptime":"0
days, 0 hours, 0 minutes, 13 seconds", "memory":"34.4 MB (%7) of
490.7 MB"}
Now check if you can access solr admin panel
http://localhost:8983/solr/
Check if solr is running, start solr if not running and then try
sudo service solr status
If you are still not able to access solr admin panel, then may you OS firewall is blocking to port 8983
use following commands to allow port 8983 in firewall
sudo firewall-cmd --zone = public --add-port = 8983 / tcp --permanent
sudo firewall-cmd --reload
For further data migration please see Migration of solr cores after solr version upgradation from 4.10.4 to 6.6.0 on Cent os 7
I have installed zookeeper on 3 machines. And set zoo.conf file. It's running with configuration. I installed solr also on these 3 machine.
Now i need to know how to run solr cloud on group of machine with number of shards?
My zookeeper configuration on all machine :-
tickTime=2000
dataDir=/var/zookeeper
clientPort=2181
initLimit=5
syncLimit=2
server.1=instance-1:2888:3888
server.2=instance-2:2889:3889
server.3=instance-3:2890:3890
zookeeper command to run it on all machine:-
bin/zkServer.sh start conf/zoo.cfg
Solr command to run it:-
bin/solr start -e cloud -z instance-1:2181,instance-2:2182,instance-3:2183 -noprompt
It's just creating 2 shards on each machine individually on default port. Not able to connect machine with each other.
I have started learning solr.I have downloaded the latest zip(5.1.0) provided by solr and run the server using bin/solr start -e cloud -noprompt.
I check that this internally calls
bin/solr start -cloud -s example/cloud/node1/solr -p 8983
bin/solr start -cloud -s example/cloud/node2/solr -p 7574 -z localhost:9983
I check that these is no config(conf/solrconfig.xml) defined in example/cloud/node1/solr so how does solr load config from the SOLR_HOME/configsets directory?
I read the documentation on several places but i am still unable to figure out the use of cloud like in 'bin/solr start -cloud -s ... ' and use of zookeeper.
Please help.
when you are working on solr cloud with zookeeper, you have to upload your solr config on zookeeper.
./bin/solr zk -upconfig -z localhost:2181,localhost:2182,localhost:2182 -n my-config -d server/solr/files/conf/
using upconfig you can upload your solr config, only have to provide path of your config directory.
You can use config name(my-config) for create core using api.
http://XXX.XXX.XXX.XXX:8983/solr/admin/collections?action=CREATE&name=irTest&numShards=3&replicationFactor=2&maxShardsPerNode=3&collection.configName=my-config
So it will create core using your config only.
Download the latest version of the Apache solr reference guide.
https://www.apache.org/dyn/closer.cgi/lucene/solr/ref-guide/apache-solr-ref-guide-5.1.pdf
check this section in the PDF.
Configuration Directories and SolrCloud
Since you are not specifying a specific configset, the default is loaded.
First, if you don't provide the -d or -n options, then the default
configuration ($SOLR_HOME/server/solr/con
figsets/data_driven_schema_configs/conf) is uploaded to ZooKeeper
using the same name as the collection
I have 3 Zookeeper servers running at server1:2181, server2:2181 and server3:2181.
I want to start 4 Solr servers at server1:8983,server2:8983,server3:8983 and server4:8983 to point to Zookeeper Ensemble above. So at server1, I run a command:
bin>solr -c -z server1:2181,server2:2181,server3:2181 -m 2g
and I received an error message:
Missing operand.
Invalid command-line option: server2:2181
Usage:.........
but if I point to one Zookeeper server such as:
bin>solr -c -z server1:2181 -m 2g
it starts successfully.
All server running in Windows.
What did I do wrong? Or does the Solr start script in Windows have an error?
You need to put your zk connection string in quotes.