How to make SolrCloud use HTTPS for creating collections? - solr

I'm setting up SolrCloud 7.2.1 on a Windows Server 2016, following the getting started guide: https://lucene.apache.org/solr/guide/7_2/getting-started-with-solrcloud.html#GettingStartedwithSolrCloud-InteractiveStartup
I have to specify the host because for some reason the SOLR_HOST variable does not have any effect. localhost gets tried all to time.
So specifying the host, everything works fine until a new collection has to be created.
For some reason I get the following error:
ERROR: Failed to create collection 'collectionname' due to: {solrurl:8984_solr=org.apache.solr.client.solrj.SolrServerException:IOException occured when talking to server at: http://solrurl:8984/solr, solrurl:8983_solr=org.apache.solr.client.solrj.SolrServerException:IOException occured when talking to server at: http://solrurl:8983/solr}
I'm a bit puzzled here because I don't understand why it uses HTTP instead of HTTPS.
I can call the URLs with HTTPS without any problems.
When creating a new collection inside the Solr web interface, the same error occures.
SOLR_SSL_ENABLED variable is set to true.
So how can I change this behaviour? Is there anything I have to do beforehand to make Solr accept changes to the solr.in.cmd and solr.in.sh?

If you want to use inter node communcation over SSL, you have to tell Solr to use that first before bringing the nodes up.
You can use the bundled zkcli tool to set the cluster property first:
server/scripts/cloud-scripts/zkcli.sh -zkhost localhost:2181 -cmd clusterprop -name urlScheme -val https

Related

SolrJ : Server refused connection

For a customer, I need to write a search engine running on Linux. I am using SolrJ and did not configure anything else so far.
I followed https://lucene.apache.org/solr/guide/7_4/using-solrj.html#common-build-systems and thus added SolrJ in the project pom.xml, and also that tutorial.
The SolR client is instanciated like :
solrClient = new HttpSolrClient.Builder(
GeneralSettings.getRootSolrPath() + "/" + getCollectionName()).
build();
But for any query or commit I keep getting org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://localhost:8983/solr/test. I read http://lucene.472066.n3.nabble.com/Default-query-error-quot-Server-refused-connection-quot-td4010806.html but I am already using the expected port.
My understanding of the java doc SolrClient ’s handle the work of connecting to and communicating with Solr, and are where most of the user configuration happens. is that I only need to import the jar and then everything will work out of the box.
But as I keep getting this "Server refused connection" error I may have to configure something, but I could not find how to configure SolrJ (use solrconfig.xml or core.properties or call System.setProperty or call an API).
Please note that Apache may be running somewhere because I used to test some sites on it.
So how to get rid of this "Server refused connection" error?
Any help or tutorial to set SolrJ up based on Solr available doc would be very much appreciated,
Edit 2018-08-12 16:10
I thought SolrJ could work like Lucene, without a server, but it looks that I missed one essential piece: installing Solr (see https://www.baeldung.com/apache-solrj). I'll give it a try and post updates.
In case it might help someone else starting with SolrJ here are the steps I did to get rid of the error mentionned in the title (actually I followed https://www.baeldung.com/apache-solrj).
Downloaded the latest binary release of Solr
Extracted it somewhere
CDed into that dir
Lauched bin/solr start from that dir
Created a core with bin/solr create -c coreName (maybe another way exists but I hadn't been able to make it work so far)
Then Solr was running and listening on port 8983, and my Java app could connect to it via SolrJ.

Jackrabbit Oak: Getting started and connect to a standalone repository via RMI

I am totally new to Jackrabbit and Jackrabbit Oak. I worked a lot with Alfresco though, another JCR compliant open-source content repo.
I want to start a standalone Jackrabbit Oak repo, then connect to it via Java code. Unfortunately the Oak documentation is quite scarce.
I checked out the Oak repo, built it with mvn clean install and then ran the standalone server (memory repository is fine for me at the moment for testing) via:
$ java -jar oak-run-1.6-SNAPSHOT.jar server
Apache Jackrabbit Oak 1.6-SNAPSHOT
Starting Oak-Memory repository -> http://localhost:8080/
13:14:38.317 [main] WARN o.a.j.s.r.d.ProtectedRemoveManager - protectedhandlers-config is missing -> DIFF processing can fail for the Remove operation if the content toremove is protected!
When I open http://localhost:8080/ I see a blank page with code like this but the html / xhtml output as source like this:
I try to connect via Java code:
JcrUtils.getRepository("http://localhost:8080");
// or
JcrUtils.getRepository("http://localhost:8080/rmi");
but getting:
Connecting to http://localhost:8080
Exception in thread "main" javax.jcr.RepositoryException: Unable to access a repository with the following settings:
org.apache.jackrabbit.repository.uri: http://localhost:8080
The following RepositoryFactory classes were consulted:
org.apache.jackrabbit.oak.jcr.OakRepositoryFactory: declined
org.apache.jackrabbit.commons.JndiRepositoryFactory: declined
Perhaps the repository you are trying to access is not available at the moment.
at org.apache.jackrabbit.commons.JcrUtils.getRepository(JcrUtils.java:223)
at org.apache.jackrabbit.commons.JcrUtils.getRepository(JcrUtils.java:263)
at Main.main(Main.java:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
(The Oak documentation is not as complete as the Jackrabbit documentation, but I am also not sure how much of Jackrabbit 2 is still valid for Oak, since it's a complete rewrite.)
I found the same question in the mailing list/Nabble, but the provided answer there does not use a remote, standalone repository but a local one running in the same servlet container and even app (just that eventually the Mongo DB / Node store is configured as remote, but that would mean that the Mongo ports would need to be open). So the app creates the repository itself, which is not my case (I got this case working fine in Oak as well).
In Jackrabbit2 (not Oak), I can simply connect via
Repository repo = new URLRemoteRepository("http://localhost:8080/rmi");
and it's working fine, but this method is not available for Oak, it seems.
Is RMI not enabled by default in Oak? Is there a different URI to use?
However, the documentation of Oak says "Oak comes with a runnable jar" and the runnable jar offers the server method to start the server, so I assume that my scenario above is a valid one.
The blank page is a result of your browser being unable to parse the<title/> tag.
Go into developer mode to see how the browser incorrectly interpreted that tag.
Incorrect interpretation of title tag
i never saw an example of jackrabbit oak working like this.. are you sure it is possible to start oak outside of your application?
How do you set up the persistent store? (which one are you going to use?).
Here is the link how you normally set up jackrabbit oak: https://jackrabbit.apache.org/oak/docs/construct.html
For example if you use MongoDB as backend (which is the most powerful), you first connect to the db via
Db db = new MongoClient(ip, port).getDB("testDB");
where ip is the ip-address of your MongoDB-server with its port. This server doesn't need to be on the same machine like your Java code is running. You can even use instead of a single MongoDB instance a Replica set.
The same is valid by using a relational db.. only if you choose the tar-file system backend you are limited to your local machine.
Then, in a second step you create a jcr based on the chosen backend (see the link)

Create Solr core in console with authentication

I have a Solr core that needs an authentication. Let's say I have a user with password password.
When I now try to create a Solr core in the console with
bin\solr create -c test
I get an HTTP Error 401.
Of course I need to specify my user. How can I add my user and password in the console?
Something like bin\solr create -c test -authentication user:password
Using Command Line scripts with BasicAuth
Add the following line to the (/opt/solr/bin) solr.in.sh or solr.in.cmd file. This example tells the bin/solr command line to to use "basic" as the type of authentication, and to pass credentials with the user-name "solr" and password "SolrRocks":
SOLR_AUTH_TYPE="basic"
SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
look:
https://lucene.apache.org/solr/guide/6_6/basic-authentication-plugin.html#BasicAuthenticationPlugin-UsingBasicAuthwithSolrJ
If you use Basic Authentication than first set in the CMD Console:
set SOLR_AUTH_TYPE=basic
set SOLR_AUTHENTICATION_OPTS="-Dbasicauth=USER:PASS"
Customize USER:PASS for your need
It has been a while so, I am not sure if you still have this issue. But this can help others who may come across this like how I did.
I am new to solr, so there could be a better way of doing this than what I have here.
Are you using zookeeper for authentication?
Assuming you are using zookeeper for authentication
First of all, I checked and I don't think there is an -authentication parameter for solr create command.
So, this is how I hacked around this issue:
Stopped solr
Started solr without zookeeper
created the core like how you tried for the test core
Then uploaded configuration of the test core to zookeeper. I followed this document - https://systemsarchitect.net/2013/04/06/painless-guide-to-solr-cloud-configuration/
Restarted zookeeper server
Started solr with -p 8983 -z zookeeper_hostname:zookeeper_port
You should be able to access your test core with authentication enabled

Carrot2 dcs webapp setup

I have been struggling with setting up Carrot2 for use PHP, on a local machine. The plan is to have Carrot2 retrieve cluster from Solr populated by Nutch. Currently Solr and Nutch are correctly configured and I have been able to access the information via Carrot2 Workbench. Carrot2-dcs-3.10.0 has been set up what I believed to be correctly deployed through the tomcat6 manager although the documentation on setting this up is horrible vague and incomplete. Changes to source-solr-attributes.xml were made according to https://sites.google.com/site/profileswapnilkulkarni/tech-talk/howtoconfigureandruncarrot2webapplicationwithsolrdocumentsource . Tomcat is set up on port 8080. The Carrot2 DCS php example example.php works and displays the test output correctly. Although, when I try to perform a cluster using localIPAddress:8080/carrot2-dcs/index.html I run into a problem. When I set document source to Solr and the query to : then click cluster I get the following error message.
HTTP Status 500 - Could not perform processing: org.apache.http.conn.HttpHostConnectException: Connection to localhost:8983 refused
type Status report
message Could not perform processing: org.apache.http.conn.HttpHostConnectException: Connection to localhost:8983 refused
description The server encountered an internal error that prevented it from fulfilling this request.
I have searched everywhere in the deployed webapp folder for carrot2 and can't find where it is getting localhost:8983 from.
Any assistance would be appreciated, thank you.
It turns out that the source-solr-attributes.xml file had an extra overridden-attributes. one was before the default block comment with the example parameters and the second was added in by me with the parameters needed for my config. Deleting one of the line so there was only one corrected the problem. Apparently with two of those it ignores the server settings and uses default values instead.

How to change the admin port using the Google App engine launcher?

It's easy to change the port of the application using the launcher. Go to Edit>>Application Settings (or alternatively Ctrl+i) and change the port. Since I was getting an error similar to the one given below, I changed the application port from 8080 to 48080. Unfortunately, I am not able to change the admin port from 8000 to 48000, because of which I am getting the following error.
google.appengine.tools.devappserver2.wsgi_server.BindError: Unable to bind localhost:8000
I also tried the method illustrated in the following image (my reference being this link):
Regardless of what I try, my web-browsers (Chrome and Firefox) return a message similar to This webpage is not available. Any ideas are appreciated.
I have seen this before, when I had a crashed instance running on the port. You may have a frozen Python script running on that port. On a Mac, I can go to Activity Monitor and kill the process. Not sure if you can do that via task manager in Windows. You may need to restart machine.

Resources