solr highlighting shows empty result - solr

Hi I am new in Solr and using Solr 7.0.0 running in windows 7.
I created a collection and indexed folder with pdf and html files residing in a folder using the following command:
> java -jar -Dc=guidanceDoc -Dauto example\exampledocs\post.jar M:\Projects\guidance\documents\*
If I write a query, I get results. However, if I turn the hl=on, I get a section for highlights without any text.
Here is the query:
http://localhost:8983/solr/guidanceDoc/select?hl.fl=_text_&hl=on&%20q=_text_:"Home%20Use"
Here is the highlight part of the result:
"highlighting":{
"M:\\Projects\\g1\\documents\\gg331681":{},
"M:\\Projects\\g1\\documents\\gg209337":{},
"M:\\Projects\\g1\\documents\\ggM380327":{},
"M:\\Projects\\g1\\documents\\gg470201":{},
"M:\\Projects\\g1\\documents\\gg507278":{},
"M:\\Projects\\g1\\documents\\gg073767":{},
"M:\\Projects\\g1\\documents\\gg380325":{},
"M:\\Projects\\g1\\documents\\gg484345":{},
"M:\\Projects\\g1\\documents\\gg259760":{}}}
How can I make it work?

Your field for a highlighting should be marked as stored=true. Since you're running Solr in cloud mode, I will recommend to use Schema API to change field definition:
curl -X POST -H 'Content-type:application/json' --data-binary '{
"replace-field":{
"name":"hl_field",
"type":"text",
"stored":true }
}' http://localhost:8983/solr/guidanceDoc/schema

Related

Status 500 error while committing changes to Solr index

I have added files to our Solr index with a curl command:
curl http://<servername>:8983/solr/<collection>/update -H \"Content-Type: text/xml\" --data-binary 1.xml
This ran fine and added the files.
On the Solr UI, in the statistics section for the collection, the "Current" symbol has changed from a green tick to a red circle (indicating there are uncommitted changes).
I'm now trying to commit the changes using the curl:
curl http://<servername>:8983/solr/<collection>/update?commit=true
However this produces the following error response:
{
"responseHeader":{
"status":500,
"QTime":6},
"error":{
"msg":"/opt/solr/solr_dir/solr-8.8.0/server/solr/inc_tickets/data/index/_qz.fdt",
"trace":"java.nio.file.AccessDeniedException: /opt/solr/solr_dir/solr-8.8.0/server/solr/inc_tickets/data/index/_qz.fdt\n\tat sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)\n\tat sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)\n\tat sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)\n\tat sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)\n\tat java.nio.channels.FileChannel.open(FileChannel.java:287)\n\tat java.nio.channels.FileChannel.open(FileChannel.java:335)\n\tat org.apache.lucene.util.IOUtils.fsync(IOUtils.java:469)\n\tat org.apache.lucene.store.FSDirectory.fsync(FSDirectory.java:331)\n\tat org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:286)\n\tat org.apache.lucene.store.NRTCachingDirectory.sync(NRTCachingDirectory.java:158)\n\tat org.apache.lucene.store.LockValidatingDirectoryWrapper.sync(LockValidatingDirectoryWrapper.java:68)\n\tat org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:5103)\n\tat org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:3461)\n\tat org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3771)\n\tat org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3729)\n\tat org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:675)\n\tat org.apache.solr.update.processor.RunUpdateProcessorFactory$RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:97)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:68)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:68)\n\tat org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1082)\n\tat org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1069)\n\tat org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:169)\n\tat org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)\n\tat org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:76)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2646)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:794)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:567)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)\n\tat org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)\n\tat org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)\n\tat org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\n\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\n\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)\n\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)\n\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)\n\tat java.lang.Thread.run(Thread.java:748)\n",
"code":500}}
The user that Solr is running under doesn't have access to the /opt/solr/solr_dir/... directory.
Give it access or change the owning user for the directory where the index is stored (and the configuration if you want to update it through the API).
You can use chmod to change access rights to a directory or chown <user>.<group> to change who the owner of the directory is. If you include -R it will change the property for the whole hierarchy.

Solr not returning highlighted results

I am using nutch 1.15 and solr 7.3, and I followed search highlight as per doc - https://lucene.apache.org/solr/guide/7_3/highlighting.html
For me, normal query for nutch solr search is working and it is returning results:
curl http://localhost:8983/solr/nutch/select?q=content:build&wt=json&rows=10000&start=0
With search highlight query I am getting same results but getting a warning.- hl.q=content:build: not found
The query with highlight params are like below - curl http://localhost:8983/solr/nutch/select?q=content:build&hl=on&hl.q=content:build&wt=json&rows=10000&start=0
See the complete response -
$ curl http://localhost:8983/solr/nutch/select?q=content:build&hl=on&hl.q=content:build&wt=json&rows=10000&start=0
-sh: 8: hl.q=content:build: not found
[3] Done(127) hl.q=content:build
[2] Done curl http://localhost:8983/solr/nutch/select?q=content:build
$ {
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"q":"content:build"}},
"response":{"numFound":2,"start":0,"docs":[
{
"digest":"ff0d20368525b3a0f14933eddb0809db",
"boost":1.0151907,
"id":"https://dummy_url",
"title":"dummy title",
"content":"dummy content",
"_version_":1691148343590256640},
{
"digest":"4fd333469ed5d83ad08eaa7ef0b779c4",
"boost":1.0151907,
"id":"https://dummy_url1",
"title":"dummy title1",
"content":"dummy content1",
"_version_":1691148343603888128}]
}}
Anyone have idea on how to resolve this? I am not getting any errors in nutch and solr logs.
You're not running the command you think you're running - & signals to the shell that the command should be run in the background, so what's effectively happening is that you're running multiple commands:
curl http://localhost:8983/solr/nutch/select?q=content:build
hl=on
hl.q=content:build
wt=json
rows=10000
start=0
This is not what you intend to do. You can either wrap your URL within quotes (") or escape the ampersands:
curl "http://localhost:8983/solr/nutch/select?q=content:build&hl=on&hl.q=content:build&wt=json&rows=10000&start=0"
# or
curl http://localhost:8983/solr/nutch/select? q=content:build\&hl=on\&hl.q=content:build\&wt=json\&rows=10000\&start=0

Solr 8.2 on OpenJDK 12 is not adding users via the API in standalone mode with security.json file

I've set up simple Apache Solr installation 8.2.0 and have managed to get it to run, however when I add the default security.json file it will not allow me to add users using the v1 API. This server is in standalone mode.
OS: Ubuntu 18.04
security.json location: /var/solr/data
I've added the default security.json file:
{
"authentication":{
"blockUnknown": true,
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="},
"realm":"My Solr users",
"forwardCredentials": false
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[{"name":"security-edit",
"role":"admin"}],
"user-role":{"solr":"admin"}
}}
And this allows for authentication using the default solr/SolrRocks account. I want to create a new user and remove the default user. When I run the API command documented here:
https://lucene.apache.org/solr/guide/8_2/basic-authentication-plugin.html
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{"set-user": {"tom" : "TomIsCool"}}'
I get this response from Solr:
{
"responseHeader":{
"status":500,
"QTime":27},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","java.nio.file.AccessDeniedException"],
"msg":"Failed persisting security.json to /var/solr/data/security.json",
"trace":"org.apache.solr.common.SolrException: Failed persisting security.json to /var/solr/data/security.json\n\tat org.apache.solr.handler.admin.SecurityConfHandlerLocal.persistConf(SecurityConfHandlerLocal.java:89)\n\tat org.apache.solr.handler.admin.SecurityConfHandler.doEdit(SecurityConfHandler.java:134)\n\tat org.apache.solr.handler.admin.SecurityConfHandler.handleRequestBody(SecurityConfHandler.java:85)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:820)\n\tat org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:786)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:546)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:505)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)\n\tat java.base/java.lang.Thread.run(Thread.java:835)\nCaused by: java.nio.file.AccessDeniedException: /var/solr/data/security.json\n\tat java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)\n\tat java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)\n\tat java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:478)\n\tat java.base/java.nio.file.Files.newOutputStream(Files.java:222)\n\tat org.apache.solr.handler.admin.SecurityConfHandlerLocal.persistConf(SecurityConfHandlerLocal.java:84)\n\t... 37 more\n",
"code":500}}
There are no other serivces on this box and I can't find anything in Google or here describing this issue. Thanks!
The /var/solr/data/security.json is owned by a different user than what Solr runs under, and the permissions set doesn't allow the Solr user to write to it.
You can check the permissions by doing ls -al /var/solr/data/security.json and comparing the owning user to the user running solr. You can see the user Solr is running under by doing ps aux | grep solr.
Changing ownership of the file can be done by issuing a chown command, or changing permissions through chmod.

How can we change value openSearcher=true from JAVA Code?

I am trying to insert data to SOLR using HardCommit. By default value of openSearcher=false in solrConfig.xml. I want to change openSearcher=true through JAVA code. Donot want to make change to SOLR.config.xml. Is there any way to do that??
Thanks
You should be able to do that with Config API. You could check current configuration by firing GET request at /config endpoint. E.g
http://localhost:8983/solr/collection-name/config
and set some property with command like this:
curl http://localhost:8983/solr/collection-name/config
-H 'Content-type:application/json' -d
'{
"set-property": {
"updateHandler.autoCommit.openSearcher": true
}
}'
This could be of course done in Java code, by using some popular HTTP client or by firing your implementation of abstract SolrRequest in SolrJ

Error when adding user for Solr Basic Authentication

When I try to add the user for the Solr Basic Authentication using the following method in curl
curl --user user:password http://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{
"set-user": {"tom" : "TomIsCool" ,
"harry":"HarrysSecret"}}'
I get the following error:
{
"responseHeader":{
"status":400,
"QTime":0},
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":"No contentStream",
"code":400}}
curl: (3) [globbing] unmatched brace in column 1
枩]?V7`-{炘9叡 t肤 ,? E'qyT咐黣]儎;衷 鈛^W褹?curl: (3) [globbing] unmatched cl
ose brace/bracket in column 13
What does this error means and how should we resolve it?
I'm using SolrCloud on Solr 6.4.2.
Regards,
Edwin
If you're using curl under Windows, this is a known issue with cmd.exe's escaping of single quotes. Use double quotes around your JSON string (or use cygwin, powershell, etc.)
curl --user user:password http://localhost:8983/solr/admin/authentication -H
"Content-type:application/json" -d "{
\"set-user\": {\"tom\" : \"TomIsCool\" ,
\"harry\":\"HarrysSecret\"}}"
The "globbing" message from curl is the hint that curl is doing something else than what you intended, and that the actual body of the request isn't getting to Solr (which is complaining about no message body being present).
You could also get around this by using stream.body in the URL and making the request from your browser.

Resources