On running the following query with
q=: and fq=(
{!parent which=type:scrap v='visibility:show AND (stock:1)'}
) we ran into the following exception :
java.lang.ArrayIndexOutOfBoundsException: 33554431\n\tat
org.apache.lucene.util.FixedBitSet.get(FixedBitSet.java:150)\n\tat
org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:284)\n\tat
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:177)\n\tat
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:148)\n\tat
org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)\n\tat
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:592)\n\tat
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:284)\n\tat
org.apache.solr.search.SolrIndexSearcher.getDocSetNC(SolrIndexSearcher.java:1264)\n\tat
org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher.java:951)\n\tat
org.apache.solr.search.SolrIndexSearcher.getProcessedFilter(SolrIndexSearcher.java:1109)\n\tat
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1630)\n\tat
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1506)\n\tat
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:586)\n\tat
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:511)\n\tat
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:227)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:144)\n\tat
org.apache.solr.core.SolrCore.execute(SolrCore.java:2006)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:204)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\n\tat`
This used to run perfectly fine before. We increased the memory for SOLR from 3000M to 4096M and ran Optimise again and things seemed to work fine. I want to understand if this can occur again and the steps we need to take to prevent it from occurring
Suggest you raise this to the lucene / solr users mailing list. See here for more details http://lucene.apache.org/solr/resources.html
Related
In DSpace 7.4, while adding new metadata to an existing item I'm not getting any suggestions in the input box, is there any configuration step I'm missing?
I can see it working in the official demo site
here
I see in dspace.log that there is this API call when I type something in the input box:
GET /server/api/core/metadatafields/search/byFieldName
so I tried this with no results:
/server/#api/core/metadatafields/search/byFieldName?query=author
The same API call in the DSpace 7 demo site returns 3 results: https://api7.dspace.org/server/#api/core/metadatafields/search/byFieldName?query=author
Also, in solr.log I see this call, which returns no hits:
2023-01-11 13:09:55.709 INFO (qtp359742806-22) [ x:search] o.a.s.c.S.Request [search] webapp=/solr path=/select params={q=*:*&fl=search.resourcetype,search.resourceid,search.uniqueid,database_status&start=0&fq=fieldName_keyword:author*&fq=&wt=javabin&version=2} hits=0 status=0 QTime=1
Maybe there is a problem with SOLR?
Since it seemed to be a SOLR problem, I tried to reindex like this: dspace index-discovery -b, thus getting this error:
java.lang.NullPointerException: Cannot invoke "org.dspace.eperson.EPerson.getID()" because the return value of "org.dspace.authorize.ResourcePolicy.getEPerson()" is null
at org.dspace.discovery.SolrServiceResourceRestrictionPlugin.additionalIndex(SolrServiceResourceRestrictionPlugin.java:95)
at org.dspace.discovery.indexobject.IndexFactoryImpl.buildDocument(IndexFactoryImpl.java:67)
at org.dspace.discovery.indexobject.InprogressSubmissionIndexFactoryImpl.buildDocument(InprogressSubmissionIndexFactoryImpl.java:46)
at org.dspace.discovery.indexobject.WorkspaceItemIndexFactoryImpl.buildDocument(WorkspaceItemIndexFactoryImpl.java:63)
at org.dspace.discovery.indexobject.WorkspaceItemIndexFactoryImpl.buildDocument(WorkspaceItemIndexFactoryImpl.java:30)
at org.dspace.discovery.SolrServiceImpl.update(SolrServiceImpl.java:165)
at org.dspace.discovery.SolrServiceImpl.indexContent(SolrServiceImpl.java:155)
at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:340)
at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:327)
at org.dspace.discovery.IndexClient.internalRun(IndexClient.java:130)
at org.dspace.scripts.DSpaceRunnable.run(DSpaceRunnable.java:104)
at org.dspace.app.launcher.ScriptLauncher.executeScript(ScriptLauncher.java:149)
at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:131)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98)
And found the solution here: https://groups.google.com/g/dspace-tech/c/ioukme4el9o.
WARNING: It's recommended to test this in a development environment first.
The problem is that there was several rows with no eperson_id and no epersongroup_id in the resourcepolicy table. So, I've deleted those rows with delete from resourcepolicy where eperson_id is null and epersongroup_id is null; and reindexed: dspace index-discovery -b.
Note: this problem seems to occur whenever the database comes from a dump of a previous DSpace version. In my case I restored a dump from a 5.6 version and migrated it to 7.4 before running into this issue.
I got a perfectly fine working SolrCloud cluster.
When I try to search for "Onesie", it gives me org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
All other requests return correct sets of documents. All nodes appear to be live.
Is the word "Onesie" cursed? What kind of magic is that?
There was a line in Solr synonyms.txt that was breaking it:
Onesie, Baby Girl Rompers & One-pieces, Baby Boy One-Pieces
I changed it to
Onesie, Baby Rompers, Baby One-Pieces
And the issue went away
I am using solr 4.3.0 for my web site search. I want to do something using solr but when I query, I get an error. In my situation I have 40000 products, and I want to excludes 1500 products with query. This is the my query
-brand-slug:reebok OR -brand-slug:nike AND
-skuCode:(01-117363 01-117364 01-117552 01-119131 01-119166 01-1J622 01-1J793 01-1M4434 01-1M9691 01-1Q279 01-1T405 01-1T865 01-2109830 01-2111116 01-2111186 01-21J625 01-21J794 01-21V019 01-2M9691 01-2M9696 01-33J793 01-519075 01-M4431 01-M7652 01-M9160 01-M9165 01-M9166 01-M9613 01-M9622 01-M9697 01200CY0001N00 01211SU0141M00 01212KU0009N00 01212KU0010N00 01212KU0025N00 01212KU0027N00 01212KU0038N00 01212KW0019N00 01212KW0020N00
....thousands of skuCodes)
If I put 670 skuCodes in their that will works good, but I use 1500 skuCodes is an error like
Solr HTTP error: OK (400)
How could I solve this problem? Thanks
What a night :) I solved my problem. Actually there was 2 problems in my system. First problem is in my tomcat server. I increase their request size with change maxHttpHeaderSize="65536". ( You could change your web server buffer size I changed my nginx conf). The other problem is about solr config. I got an error like 'too many boolean clauses'. If you get this error, you could change maxBooleanClauses in solrconfig.xml. After restart my tomcat server everything was ok.
I have been working on Solr and running some load tests on it. After some point, I keep getting
Nov 29, 2012 3:34:43 PM org.apache.solr.common.SolrException log
SEVERE: null:java.lang.IllegalStateException: Form too large275768>200000
at org.eclipse.jetty.server.Request.extractParameters(Request.java:279)
at org.eclipse.jetty.server.Request.getParameterMap(Request.java:705)
at org.apache.solr.request.ServletSolrParams.<init>(ServletSolrParams.java:29)
at org.apache.solr.servlet.StandardRequestParser.parseParamsAndFillStreams(SolrRequestParsers.java:394)
at org.apache.solr.servlet.SolrRequestParsers.parse(SolrRequestParsers.java:115)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
at org.eclipse.jetty.server.Server.handle(Server.java:351)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
at java.lang.Thread.run(Unknown Source)
Basically I made searches on google and stackoverflow too, and all I could find was this and applying the solutions there didnt helped at all..
I have tried to modify that value from org.apache.solr.client.solrj.embedded.JettySolrRunnertoo but even changing value from that file didnt helped at all.
anyone knows how to change max allowed form size for an embedded Jetty?
After checking the source code of Solr, I found one place where I can set the form size. The class I have modified is org.apache.solr.client.solrj.embedded.JettySolrRunner.java , basically adding some large number for the form size...
although it works, I am still confused why I cant set this value via config files
Periodically, I'm getting problems with my Tomcat 6 cluster (2 nodes). One of the nodes would just go haywire and generate a ton of logs repeating the following:
Aug 25, 2009 11:44:10 AM org.apache.catalina.ha.session.DeltaRequest reset
SEVERE: Unable to remove element
java.util.NoSuchElementException
at java.util.LinkedList.remove(LinkedList.java:788)
at java.util.LinkedList.removeFirst(LinkedList.java:134)
at org.apache.catalina.ha.session.DeltaRequest.reset(DeltaRequest.java:201)
at org.apache.catalina.ha.session.DeltaRequest.execute(DeltaRequest.java:195)
at org.apache.catalina.ha.session.DeltaManager.handleSESSION_DELTA(DeltaManager.java:1364)
at org.apache.catalina.ha.session.DeltaManager.messageReceived(DeltaManager.java:1320)
at org.apache.catalina.ha.session.DeltaManager.messageDataReceived(DeltaManager.java:1083)
at org.apache.catalina.ha.session.ClusterSessionListener.messageReceived(ClusterSessionListener.java:87)
at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:916)
at org.apache.catalina.ha.tcp.SimpleTcpCluster.messageReceived(SimpleTcpCluster.java:897)
at org.apache.catalina.tribes.group.GroupChannel.messageReceived(GroupChannel.java:264)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79)
at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.messageReceived(TcpFailureDetector.java:110)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79)
at org.apache.catalina.tribes.group.ChannelInterceptorBase.messageReceived(ChannelInterceptorBase.java:79)
at org.apache.catalina.tribes.group.ChannelCoordinator.messageReceived(ChannelCoordinator.java:241)
at org.apache.catalina.tribes.transport.ReceiverBase.messageDataReceived(ReceiverBase.java:225)
at org.apache.catalina.tribes.transport.nio.NioReplicationTask.drainChannel(NioReplicationTask.java:188)
at org.apache.catalina.tribes.transport.nio.NioReplicationTask.run(NioReplicationTask.java:91)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
That's the only thing that it shows. The other node in the cluster is still active at this time. There's nothing to do but to restart. The large amount of logs has caused disk space issues more than a couple of times too.
Does anybody have any idea what's wrong here?
Thanks!
Wong
Appears to be a bug in Tomcat 6. If you look at the source at:
http://www.java2s.com/Open-Source/Java-Document/Sevlet-Container/apache-tomcat-6.0.14/org/apache/catalina/ha/session/DeltaRequest.java.htm (line 225)
you'll see that the reset() method can potentially throw this exception. I suggest that you contact the Tomcat developers regarding this issue.