I'm doing a simple partial update scenario which worked with version 6.x and 7.x of Solr. After upgdrading both Solr and Solrj to 8.8, I'm getting the following exception:
2021-02-23 14:57:58.201 ERROR (qtp-459670553-28) [ x:core1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: TransactionLog doesn't know how to serialize class org.apache.lucene.document.LazyDocument$LazyField; try implementing ObjectResolver?
at org.apache.solr.update.TransactionLog$1.resolve(TransactionLog.java:100)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:266)
at org.apache.solr.common.util.JavaBinCodec$BinEntryWriter.put(JavaBinCodec.java:441)
at org.apache.solr.common.ConditionalKeyMapWriter$EntryWriterWrapper.put(ConditionalKeyMapWriter.java:44)
at org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:101)
at org.apache.solr.common.MapWriter$EntryWriter.lambda$getBiConsumer$0(MapWriter.java:161)
at org.apache.solr.common.MapWriter$EntryWriter$$Lambda$548/0000000000000000.accept(Unknown Source)
at org.apache.solr.common.SolrInputDocument.lambda$writeMap$0(SolrInputDocument.java:59)
at org.apache.solr.common.SolrInputDocument$$Lambda$549/0000000000000000.accept(Unknown Source)
.....
solrj code is just similar to the sample provided here and was working before upgrade. The operation is 'add' with a simple integer field for a document whose id is provided.
Note that this is different from a previous question on stackoverflow, since I'm passing simple integer field and on solr/lucene side it's replaced with org.apache.lucene.document.LazyDocument$LazyField.
Seems to be a bug in Solr https://issues.apache.org/jira/browse/SOLR-13034 to be fixed in the next version of solr 8 (8.9).
Until it's released the workaround is to set <enableLazyFieldLoading>false</enableLazyFieldLoading> in solrconfig.xml
Related
When trying to run a SOLR query,
SELECT count(*) from products;
we are seeing the below exception from solr server, ours is a SOLR cloud setup,
SOLR version - solr-8.8.2-PATCH2
solr-solrj-8.8.2 version
Complete Stack Trace i have mentioned below,
2023-02-09 14:21:34.824 ERROR (qtp1209411469-15) [c:products s:shard3 r:core_node12 x:otmm_shard3_replica_n10] o.a.s.s.HttpSolrCall java.lang.RuntimeException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.solr.handler.sql.SolrRules
at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:746)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:592)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:357)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:201)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191)
at org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandler.java:177)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:386)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.solr.handler.sql.SolrRules
at org.apache.solr.handler.sql.SolrTableScan.register(SolrTableScan.java:72)
at org.apache.calcite.plan.AbstractRelOptPlanner.onNewClass(AbstractRelOptPlanner.java:239)
at org.apache.calcite.plan.volcano.VolcanoPlanner.onNewClass(VolcanoPlanner.java:464)
at org.apache.calcite.plan.AbstractRelOptPlanner.registerClass(AbstractRelOptPlanner.java:230)
at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1224)
at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:589)
at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:604)
at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:84)
at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:268)
at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1132)
at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:589)
at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:604)
at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:84)
at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:268)
at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1132)
at org.apache.calcite.plan.volcano.VolcanoPlanner.setRoot(VolcanoPlanner.java:265)
at org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:262)
at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:331)
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:166)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:208)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:642)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:508)
at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:478)
at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:556)
at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at org.apache.solr.client.solrj.io.stream.JDBCStream.open(JDBCStream.java:278)
at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:52)
at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:465)
at org.apache.solr.client.solrj.io.stream.TupleStream.writeMap(TupleStream.java:82)
at org.apache.solr.common.util.JsonTextWriter.writeMap(JsonTextWriter.java:164)
at org.apache.solr.common.util.TextWriter.writeMap(TextWriter.java:216)
at org.apache.solr.common.util.TextWriter.writeVal(TextWriter.java:69)
at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:153)
at org.apache.solr.common.util.JsonTextWriter.writeNamedListAsMapWithDups(JsonTextWriter.java:387)
at org.apache.solr.common.util.JsonTextWriter.writeNamedList(JsonTextWriter.java:293)
at org.apache.solr.response.JSONWriter.writeResponse(JSONWriter.java:73)
at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:66)
at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:890)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:583)
... 40 more
Kindly let me know what is being wrong with this. Also am adding additional lines because its compalining me add more details.
Iam using Solrj, and i have a application it supports only JDBC and iam in a situation to use only JDBC solr using SOLRJ,
Its basically a BIRT reporting tool.
Where we can define SQL and output would be automatically mapped to the report.
This is what we are trying to do
Added below in solr.in.sh
SOLR_OPTS="$SOLR_OPTS -Dsolr.modules=sql"
And in solr admin, can able to see,
Args in Systemdashboard,
-Dsolr.modules=sql
But still not working
{
"responseHeader":{
"status":0,
"QTime":2},
"config":{"requestHandler":{"/export":{
"class":"solr.ExportHandler",
"useParams":"_EXPORT",
"components":["query"],
"invariants":{
"rq":"{!xport}",
"distrib":false},
"name":"/export",
"_useParamsExpanded_":{"_EXPORT":"[NOT AVAILABLE]"},
"_effectiveParams_":{
"distrib":"false",
"rq":"{!xport}"}}}}}
Edit: I just realized you had linked to the wrong docs for your version. The way you are trying to add modules was not available until 9.0 (https://issues.apache.org/jira/browse/SOLR-15914) 8.8 docs are at: https://solr.apache.org/guide/8_8/ Note that until recently this feature was called Parallel Sql https://solr.apache.org/guide/8_8/parallel-sql-interface.html
For 8.8 you shouldn't need to configure modules since at that time /sql was an implicitly loaded request handler.
https://solr.apache.org/guide/8_8/implicit-requesthandlers.html
You may need to verify if the implicit handlers have been (mis)configured via the request parameters API (https://solr.apache.org/guide/8_8/implicit-requesthandlers.html#how-to-edit-implicit-handler-paramsets)
Below refers to the 9.x versions of solr
java.lang.NoClassDefFoundError: Could not initialize class org.apache.solr.handler.sql.SolrRules
Probably indicates that you have not successfully loaded the sql module. Normally one enables modules at the very bottom of solr.in.sh not solr.sh as you said in your comments. solr.in.sh is the place where Solr intends for you to set up environment variables. One really shouldn't ever need to modify solr.sh directly, and doing so may make upgrades difficult in the future.
Check that there isn't another set of enablements in solr.in.sh that is overwriting what you've tried to do in solr.sh. Also check that you aren't enabling any other modules in other ways (several methods are shown here: https://solr.apache.org/guide/solr/latest/configuration-guide/solr-modules.html). You should pick one way of enabling modules (sysprops, solr.in.sh, solr.xml or solrconfig.xml) and then enable all modules that way to avoid having to understand any complicated precedence logic if possible. I don't know the precedence order, but I can probably figure it out if you have other modules and absolutely can't avoid using more than one method.
Also, you still haven't told us where you got the version ending in -PATCH2. This version spec sounds like you are working with some folks who are compiling their own custom Solr, so you should be sure to understand what they've changed in case they've customized something about how or where Solr loads jar files (not very likely, but one never knows).
In DSpace 7.4, while adding new metadata to an existing item I'm not getting any suggestions in the input box, is there any configuration step I'm missing?
I can see it working in the official demo site
here
I see in dspace.log that there is this API call when I type something in the input box:
GET /server/api/core/metadatafields/search/byFieldName
so I tried this with no results:
/server/#api/core/metadatafields/search/byFieldName?query=author
The same API call in the DSpace 7 demo site returns 3 results: https://api7.dspace.org/server/#api/core/metadatafields/search/byFieldName?query=author
Also, in solr.log I see this call, which returns no hits:
2023-01-11 13:09:55.709 INFO (qtp359742806-22) [ x:search] o.a.s.c.S.Request [search] webapp=/solr path=/select params={q=*:*&fl=search.resourcetype,search.resourceid,search.uniqueid,database_status&start=0&fq=fieldName_keyword:author*&fq=&wt=javabin&version=2} hits=0 status=0 QTime=1
Maybe there is a problem with SOLR?
Since it seemed to be a SOLR problem, I tried to reindex like this: dspace index-discovery -b, thus getting this error:
java.lang.NullPointerException: Cannot invoke "org.dspace.eperson.EPerson.getID()" because the return value of "org.dspace.authorize.ResourcePolicy.getEPerson()" is null
at org.dspace.discovery.SolrServiceResourceRestrictionPlugin.additionalIndex(SolrServiceResourceRestrictionPlugin.java:95)
at org.dspace.discovery.indexobject.IndexFactoryImpl.buildDocument(IndexFactoryImpl.java:67)
at org.dspace.discovery.indexobject.InprogressSubmissionIndexFactoryImpl.buildDocument(InprogressSubmissionIndexFactoryImpl.java:46)
at org.dspace.discovery.indexobject.WorkspaceItemIndexFactoryImpl.buildDocument(WorkspaceItemIndexFactoryImpl.java:63)
at org.dspace.discovery.indexobject.WorkspaceItemIndexFactoryImpl.buildDocument(WorkspaceItemIndexFactoryImpl.java:30)
at org.dspace.discovery.SolrServiceImpl.update(SolrServiceImpl.java:165)
at org.dspace.discovery.SolrServiceImpl.indexContent(SolrServiceImpl.java:155)
at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:340)
at org.dspace.discovery.SolrServiceImpl.updateIndex(SolrServiceImpl.java:327)
at org.dspace.discovery.IndexClient.internalRun(IndexClient.java:130)
at org.dspace.scripts.DSpaceRunnable.run(DSpaceRunnable.java:104)
at org.dspace.app.launcher.ScriptLauncher.executeScript(ScriptLauncher.java:149)
at org.dspace.app.launcher.ScriptLauncher.handleScript(ScriptLauncher.java:131)
at org.dspace.app.launcher.ScriptLauncher.main(ScriptLauncher.java:98)
And found the solution here: https://groups.google.com/g/dspace-tech/c/ioukme4el9o.
WARNING: It's recommended to test this in a development environment first.
The problem is that there was several rows with no eperson_id and no epersongroup_id in the resourcepolicy table. So, I've deleted those rows with delete from resourcepolicy where eperson_id is null and epersongroup_id is null; and reindexed: dspace index-discovery -b.
Note: this problem seems to occur whenever the database comes from a dump of a previous DSpace version. In my case I restored a dump from a 5.6 version and migrated it to 7.4 before running into this issue.
I'm setting up my first Solr server via docker using solr:8.11.1-slim. I am gonna use the schema API to set up the schema for my core whose name is 'products'.
While reading the docs there seems to be false info on the docs for field types:
https://solr.apache.org/guide/8_11/field-types-included-with-solr.html
vs.
https://solr.apache.org/guide/8_11/schema-api.html
I followed the first guide to get info on what field types I can specify and am trying to send requests based on the second doc such as this:
{ 'add-field': { "name":"latlong", "type":"LatLongPointSpatialField", "multiValued":False, "stored":True, 'indexed': True } },
but Solr gives me back errors such as:
org.apache.solr.api.ApiBag$ExceptionWithErrObject: error processing commands, errors: [{add-field={name=latlong, type=LatLongPointSpatialField, multiValued=false, stored=true, indexed=true}, errorMessages=[Field 'latlong': Field type 'LatLongPointSpatialField' not found
So what gives? Am I misreading the docs or are they wrong or is something wrong with the solr 8.11.1 image in docker? Why does it not accept the field types I'm providing?
Thanks for your help ahead of time.
I'm doing a simple partial update scenario which worked with version 6.x and 7.x of Solr. After upgdrading both Solr and Solrj to 8.8, I'm getting the following exception:
2021-02-23 14:57:58.201 ERROR (qtp-459670553-28) [ x:core1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: TransactionLog doesn't know how to serialize class org.apache.lucene.document.LazyDocument$LazyField; try implementing ObjectResolver?
at org.apache.solr.update.TransactionLog$1.resolve(TransactionLog.java:100)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:266)
at org.apache.solr.common.util.JavaBinCodec$BinEntryWriter.put(JavaBinCodec.java:441)
at org.apache.solr.common.ConditionalKeyMapWriter$EntryWriterWrapper.put(ConditionalKeyMapWriter.java:44)
at org.apache.solr.common.MapWriter$EntryWriter.putNoEx(MapWriter.java:101)
at org.apache.solr.common.MapWriter$EntryWriter.lambda$getBiConsumer$0(MapWriter.java:161)
at org.apache.solr.common.MapWriter$EntryWriter$$Lambda$548/0000000000000000.accept(Unknown Source)
at org.apache.solr.common.SolrInputDocument.lambda$writeMap$0(SolrInputDocument.java:59)
at org.apache.solr.common.SolrInputDocument$$Lambda$549/0000000000000000.accept(Unknown Source)
.....
solrj code is just similar to the sample provided here and was working before upgrade. The operation is 'add' with a simple integer field for a document whose id is provided.
Note that this is different from a previous question on stackoverflow, since I'm passing simple integer field and on solr/lucene side it's replaced with org.apache.lucene.document.LazyDocument$LazyField.
Seems to be a bug in Solr https://issues.apache.org/jira/browse/SOLR-13034 to be fixed in the next version of solr 8 (8.9).
Until it's released the workaround is to set <enableLazyFieldLoading>false</enableLazyFieldLoading> in solrconfig.xml
What is the right syntax if my query is formulated incorrectly in the link above for Apache 3.5 SOLR and do I have to enable anything specific in solrconfig.xml and schema.xml
Using Apache SOLR 3.5 and receiving a ParseException Uknown function termfreq in FunctionQuery(tf(text,amplifiers)'
http://localhost:8983/solr/select/?fl=score,documentPageId&defType=func&q=tf%28text,amplifiers%29
I am following the syntax on other websites because I don't know how to do it for the documentation on the wiki --> http://wiki.apache.org/solr/FunctionQuery
It won't work, the function query tf(field, term) that you are attempting to use is not available in 3.5, browse through ValueSourceParser if you want to double-check. You need to get Solr 4.x nightly build - Solr Nightly Build from trunk & use it, but beware Solr 4.x is not stable & released yet, there will be a significant level of API changes compared to 3.5.
If you are interested in poking into the code, you could for instance, if you are using Maven modify pom.xml to get the atrifacts from Trunk and browse the source code starting from ValueSourceParser that should let you know if those relevance functions exist & how their implementation is.
For Ex: You will see parsers related to the term vector function queries,
// From Solr 4 `ValueSourceParser` Trunk Source Code
addParser("tf", new ValueSourceParser() {
#Override
public ValueSource parse(FunctionQParser fp) throws ParseException {
TInfo tinfo = parseTerm(fp);
return new TFValueSource(tinfo.field, tinfo.val, tinfo.indexedField, tinfo.indexedBytes);
}
});