I created multivalued field in schema.xml:
<field name="path" type="pint" multiValued="true" indexed="true" stored="true"/>
I created my own search component class like this:
public class CustomComponent extends SearchComponent {
private static final Logger LOG = LoggerFactory.getLogger(CustomComponent.class);
#Override
public void prepare(ResponseBuilder rb) throws IOException {
}
#Override
public void process(ResponseBuilder rb) throws IOException {
LOG.info("CustomComponent running ---");
SolrParams params = rb.req.getParams();
CoreContainer coreContainer = rb.req.getCore().getCoreContainer();
SolrCore solrCore = coreContainer.getCore("example_core");
SolrIndexSearcher categorySearcher = solrCore.getSearcher().get();
IndexReader categoryReader = categorySearcher.getIndexReader();
String pathId = params.get("pathId"); //3
FieldType path = solrCore.getLatestSchema().getField("path").getType();
StandardQueryParser standardQueryParser = new StandardQueryParser();
standardQueryParser.setAnalyzer(path.getQueryAnalyzer());
Query q = standardQueryParser.parse(pathId, "path");
DocList docList = searcher.getDocList(q, null, null, 0, 1000, 1000);
LOG.info(docListsize()); // returns 0
//even if there is a document which has field with such value
}
}
What is wrong here? Is there a way to search on multivalued field? Thanks for answer in advance.
Related
I am trying to add the params below (qf,bq) in a Solr query generated by Spring Data Solr.
Solr parameters are :
qf => Spring Data Solr Method?
bq => Spring Data Solr Method?
I was able to find the methods below
fq => addFilterQuery
fl => addProjectionOnField
defType => setDefType
qt => setRequestHandler
I saw an open issue qf https://jira.spring.io/browse/DATASOLR-153
How can i add the qf and bq params to Solr query built with Spring Data Solr.
Thanks
You can use the SolrCallback on template level to access the SolrClient and execute the query from there or register your own QueryParser for a custom query type.
Maybe something like:
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client);
template.registerQueryParser(EdismaxQuery.class, new EdisMaxQueryParser());
return template;
}
class EdismaxQuery extends SimpleQuery {
// ... add stuff you need. Maybe `autoRelax`
}
class EdisMaxQueryParser extends QueryParserBase<EdismaxQuery> {
DefaultQueryParser defaultQueryParser = new DefaultQueryParser();
#Override
public SolrQuery doConstructSolrQuery(EdismaxQuery source) {
// just use the default parser to construct the query string in first place.
SolrQuery target = defaultQueryParser.constructSolrQuery(source);
// add missing parameters
target.add("defType", "edismax");
target.add("qf", source....);
return target;
}
}
There're some changes in Spring Data Solr API 4.0, so you may need to change how you registered your own QueryParser a bit.
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client);
solrTemplate.registerQueryParser(EdismaxQuery.class, new EdisMaxQueryParser(new SimpleSolrMappingContext()));
return template;
}
public static class EdismaxQuery extends SimpleQuery {
private String defaultField;
private String minimumShouldMatch;
private String boostQuery;
private String queryField;
public EdismaxQuery(String queryString) {
super(queryString);
}
//... typical getter/setter
}
public static class EdisMaxQueryParser extends QueryParserBase<AbstractQueryDecorator> {
private final DefaultQueryParser defaultQueryParser;
public EdisMaxQueryParser(MappingContext<? extends SolrPersistentEntity<?>, SolrPersistentProperty> mappingContext) {
super(mappingContext);
defaultQueryParser = new DefaultQueryParser(mappingContext);
}
#Override
public SolrQuery doConstructSolrQuery(AbstractQueryDecorator queryDecorator, Class<?> domainType) {
// for some reason the API wrapped our query object with NamedObjectsQuery, so we need to unwrapped/get our actual query object first
EdismaxQuery query = (EdismaxQuery) queryDecorator.getDecoratedQuery();
// use defaultQueryParser to populate basic query parameters
SolrQuery solrQuery = defaultQueryParser.doConstructSolrQuery(query, domainType);
// set our own 'extra' parameter
if (query.getDefaultField() != null) {
solrQuery.add("df", query.getDefaultField());
}
if (query.getMinimumShouldMatch() != null) {
solrQuery.add("mm", query.getMinimumShouldMatch());
}
if (query.getQueryField() != null) {
solrQuery.add("qf", query.getQueryField());
}
if (query.getBoostQuery() != null) {
solrQuery.add("bq", query.getBoostQuery());
}
//...
return target;
}
}
Here's how you query with new EdismaxQuery object
EdismaxQuery query = new EdismaxQuery("hello world");
query.setDefType("edismax");
query.setRows(3);
query.setQueryField("text^2");
query.setMinimumShouldMatch("30%");
query.setBoostQuery("date:[NOW/DAY-1YEAR TO NOW/DAY]");
Page<ResultBean> results = solrTemplate.query("collection", query, ResultBean.class);
In order to avoid:
org.springframework.data.solr.core.QueryParserBase$NamedObjectsQuery
cannot be cast to EdismaxQuery
EdisMaxQueryParser should look like this:
class EdisMaxQueryParser extends QueryParserBase {
#Override
public SolrQuery doConstructSolrQuery(SolrDataQuery source) {
// your stuff
}
}
If you are going to add static qf expression to each select query it could be done in solrconfig.xml:
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
...
</lst>
<lst name="appends">
<str name="defType">edismax</str>
<str name="qf">offerId^100 vendorCode^100</str>
</lst>
...
</requestHandler>
you can use edismax library for spring-solr https://github.com/KmSYS/edismax-solr-spring
But to overcome the problem of overwrite the registered query parsers at afterPropertiesSet() you need to add following bean at config class,
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client) {
#Override
public void afterPropertiesSet() {
super.afterPropertiesSet();
registerQueryParser(SimpleEdismaxQuery.class, new EdisMaxQueryParser(new SimpleSolrMappingContext()));
}
};
template.afterPropertiesSet();
return template;
}
Also, sample code at https://github.com/KmSYS/edismax-solr-spring-sample
http://www.kmsys.tech/solr/edixmax-query-parser-template.html
Whenever I try to query Solr using a Spring data repository I get the following exception:
Exception in thread "main" java.lang.IllegalArgumentException: [Assertion failed] - this argument is required; it must not be null
at org.springframework.util.Assert.notNull(Assert.java:112)
at org.springframework.util.Assert.notNull(Assert.java:123)
at org.springframework.data.solr.core.convert.MappingSolrConverter$SolrPropertyValueProvider.readValue(MappingSolrConverter.java:317)
at org.springframework.data.solr.core.convert.MappingSolrConverter$SolrPropertyValueProvider.readCollection(MappingSolrConverter.java:423)
at org.springframework.data.solr.core.convert.MappingSolrConverter$SolrPropertyValueProvider.readValue(MappingSolrConverter.java:331)
at org.springframework.data.solr.core.convert.MappingSolrConverter$SolrPropertyValueProvider.readValue(MappingSolrConverter.java:308)
at org.springframework.data.solr.core.convert.MappingSolrConverter$SolrPropertyValueProvider.getPropertyValue(MappingSolrConverter.java:294)
at org.springframework.data.solr.core.convert.MappingSolrConverter.getValue(MappingSolrConverter.java:147)
at org.springframework.data.solr.core.convert.MappingSolrConverter$1.doWithPersistentProperty(MappingSolrConverter.java:134)
at org.springframework.data.solr.core.convert.MappingSolrConverter$1.doWithPersistentProperty(MappingSolrConverter.java:126)
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:257)
at org.springframework.data.solr.core.convert.MappingSolrConverter.read(MappingSolrConverter.java:126)
at org.springframework.data.solr.core.convert.MappingSolrConverter.read(MappingSolrConverter.java:113)
at org.springframework.data.solr.core.convert.MappingSolrConverter.read(MappingSolrConverter.java:88)
at org.springframework.data.solr.core.SolrTemplate.convertSolrDocumentListToBeans(SolrTemplate.java:404)
at org.springframework.data.solr.core.SolrTemplate.convertQueryResponseToBeans(SolrTemplate.java:396)
at org.springframework.data.solr.core.SolrTemplate.queryForPage(SolrTemplate.java:276)
at org.springframework.data.solr.repository.query.AbstractSolrQuery$AbstractQueryExecution.executeFind(AbstractSolrQuery.java:312)
at org.springframework.data.solr.repository.query.AbstractSolrQuery$CollectionExecution.execute(AbstractSolrQuery.java:335)
at org.springframework.data.solr.repository.query.AbstractSolrQuery.execute(AbstractSolrQuery.java:129)
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:323)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:98)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:262)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:95)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at $Proxy15.findByTitleStartingWith(Unknown Source)
at foo.bar.Application.main(Application.java:46)
Document:
public class Product {
#Id
#Field
private String id;
#Field
private String description;
#Field
private String title;
// getter/setter
}
Repository:
public interface ProductRepository extends SolrCrudRepository<Product, String> {
List<Product> findByTitleStartingWith(String title);
}
Configuration + Test:
#ComponentScan
#EnableSolrRepositories("foo.bar.repository")
public class Application {
#Bean
public SolrServer solrServer() {
return new HttpSolrServer("http://localhost:8983/solr");
}
#Bean
public SolrTemplate solrTemplate(SolrServer server) throws Exception {
return new SolrTemplate(server);
}
public static void main(String[] args) {
ApplicationContext context = new AnnotationConfigApplicationContext(Application.class);
ProductRepository repository = context.getBean(ProductRepository.class);
Product product = new Product();
product.setTitle("Foo title");
product.setDescription("Bar description");
product.setId(UUID.randomUUID().toString());
repository.save(product);
List<Product> list = repository.findByTitleStartingWith("Foo"); // <-- error
System.out.println("result: " + list);
}
}
I Found the solution myself:
I was using the default field configuration for Solr (schema.xml). This configuration includes the fields title and description by default:
<field name="title" type="text_general" indexed="true" stored="true" multiValued="true"/>
<field name="description" type="text_general" indexed="true" stored="true"/>
By default title is a multiValued field. This caused an error when Spring data tried to map the fields back to my Product class.
Removing multiValued="true" (or setting it to false) from the title field solved the problem.
I have indexed pojos in solr successfully but while reading from solr my output is wrong.
Maybe the way I am printing is not correct. Can anyone tell me how to solve this issue?
My code is as follows:
List<SampleDocument> foundDocuments = response.getBeans(SampleDocument.class);
for(SampleDocument docs:foundDocuments) {
System.out.println(docs.getTitle());
}
After printing the output I get is all Null.
SampleDocument.java
package solrobj.Asolrobj;
import java.util.List;
import org.apache.solr.client.solrj.beans.Field;
public class SampleDocument {
private int id;
private String title;
public SampleDocument() {
// required for solrj to make an instance
}
public SampleDocument(int id, String title) {
this.setId(id);
this.title = title;
}
public String getTitle() {
return title;
}
#Field("title")
public void setTitle(String title) {
this.title = title;
}
public int getId() {
return id;
}
#Field("tid")
public void setId(int id) {
this.id = id;
}
}
Schema.xml
<field name="tid" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="title" type="text_general" indexed="true" stored="true" multiValued="true"/>
error
I have the below bean class for defining my solrInputDocument
public class VenueDocumentSolr extends SolrInputDocument {
#Field
private int id;
#Field
private String uid;
...
}
And I try to get the results using the below code:
SolrQuery query = new SolrQuery();
query.setQuery(SearchForRestaurants);
QueryResponse rsp = SolrUtil.issueSolrQuery(query);
for (SolrDocument s : rsp.getResults())
System.out.println(s);
List<VenueDocumentSolr> beans = rsp.getBeans(VenueDocumentSolr.class);
the above code works sometimes and throws the below Exception rest of the time.
I cross checked and added missing fields to my bean class. But of no use. I still get the error :(
org.apache.solr.client.solrj.beans.BindingException: Could not instantiate object of class com.zvents.common.entities.solr.VenueDocumentSolr
at org.apache.solr.client.solrj.beans.DocumentObjectBinder.getBean(DocumentObjectBinder.java:68)
at org.apache.solr.client.solrj.beans.DocumentObjectBinder.getBeans(DocumentObjectBinder.java:47)
at org.apache.solr.client.solrj.response.QueryResponse.getBeans(QueryResponse.java:480)
at com.zvents.webapp.api.DeleteData.QueryAndUpdateVenuesForSearch(DeleteData.java:117)
.........
Quoting a part of the Schema
<int name="has_images">*</int>
<arr name="cuisine">
<str>*</str>
</arr>
<arr name="venue_type">
<int>*</int>
</arr>
<double name="location_0_latLon">*</double>
<float name="venue_imp">*</float>
<date name="last_indexed">*</date>
Quoting applicable fields from bean class
#Field
private int has_images;
#Field
private List<String> cuisine;
#Field
private List<Integer> venue_type;
#Field
private double location_0_latLon;
#Field
private float venue_imp;
#Field
private String last_indexed;
I did it in following way:
DocumentObjectBinder binder = new DocumentObjectBinder();
beanlist = binder.getBeans(PortalBean.class, list);
package hello;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import org.apache.solr.client.solrj.SolrClient;
import org.apache.solr.client.solrj.SolrQuery;
import org.apache.solr.client.solrj.SolrServerException;
import org.apache.solr.client.solrj.beans.DocumentObjectBinder;
import org.apache.solr.client.solrj.impl.HttpSolrClient;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
import org.apache.solr.common.SolrDocumentList;
import org.springframework.web.bind.annotation.CrossOrigin;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
#RestController
public class SearchOnNameController {
#CrossOrigin(origins = "*")
#RequestMapping("search")
public List<PortalBean> searchName(#RequestParam(value = "query", defaultValue = "*") String name) {
String urlString = "http://localhost:8983/solr/kislay";
SolrClient solr = new HttpSolrClient(urlString);
SolrQuery query = new SolrQuery();
// query.addField("name");
query.setQuery("name:" + "*" + name + "*");
PortalBean bean = null;
List<PortalBean> beanlist = new ArrayList<PortalBean>();
try {
QueryResponse response = solr.query(query);
SolrDocumentList list = response.getResults();
DocumentObjectBinder binder = new DocumentObjectBinder();
beanlist = binder.getBeans(PortalBean.class, list);
} catch (SolrServerException | IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return beanlist;
}
}
The bean is mapped as :
package hello;
import org.apache.lucene.document.TextField;
import org.apache.solr.client.solrj.beans.Field;
import org.apache.solr.schema.StrField;
public class PortalBean {
private String id;
private String name;
private String image;
private String description;
private String branding;
private double rating;
private double setup_fee;
private String transaction_fees;
private String how_to_url;
private String [] currencies;
public String getId() {
return id;
}
#Field("id")
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
#Field("name")
public void setName(String name) {
this.name = name;
}
public String getImage() {
return image;
}
#Field("image")
public void setImage(String image) {
this.image = image;
}
public String getDescription() {
return description;
}
#Field("description")
public void setDescription(String description) {
this.description = description;
}
public String getBranding() {
return branding;
}
#Field("branding")
public void setBranding(String branding) {
this.branding = branding;
}
public double getRating() {
return rating;
}
#Field("rating")
public void setRating(double rating) {
this.rating = rating;
}
public double getSetup_fee() {
return setup_fee;
}
#Field("setup_fee")
public void setSetup_fee(double setup_fee) {
this.setup_fee = setup_fee;
}
public String getTransaction_fees() {
return transaction_fees;
}
#Field("transaction_fees")
public void setTransaction_fees(String transaction_fees) {
this.transaction_fees = transaction_fees;
}
public String getHow_to_url() {
return how_to_url;
}
#Field("how_to_url")
public void setHow_to_url(String how_to_url) {
this.how_to_url = how_to_url;
}
public String [] getCurrencies() {
return currencies;
}
#Field("currencies")
public void setCurrencies(String [] currencies) {
this.currencies = currencies;
}
}
Also please note that managed-schema file need to have proper definition of the fields:
example as per my use case:
<field name="branding" type="string"/>
<field name="currencies" type="strings"/>
<field name="description" type="string"/>
<field name="how_to_url" type="string"/>
<field name="id" type="string" multiValued="false" indexed="true" required="true" stored="true"/>
<field name="image" type="string"/>
<field name="name" type="string"/>
<field name="rating" type="tdouble"/>
<field name="setup_fee" type="tdouble"/>
<field name="transaction_fee" type="string"/>
Unfortunately you do not post all of your pojo and all of your schema, so I can only guess.
The only thing I see, from the code you posted is that you are trying to let solr do some sort of conversion for you.
That is for
#Field
private String last_indexed;
And
<date name="last_indexed">*</date>
Solr will not convert a Date to a String for you, just will not. But without a full schema and the full code of your Bean, I cannot give you a full answer.
Other things I noted from the code you post
why are you putting * within the elements of your schema as far as I know that does not have any effect
why are you extending SolrInputDocument? You do not need to. #Field is designed so that you do not need to subclass anything, but may use your model's POJOs right away.
My error was: BindingException: Could not instantiate object
I added a no variable constructor to the POJO and it started to work.
public class MyItem {
String id;
#Field("id")
public void setId(String id) {
this.id = id;
}
public String getId() {
return id;
}
public MyItem() {
}
}
To get to here the only fields in my schema were:
<field name="_version_" type="long" indexed="true" stored="true"/>
<field name="_root_" type="string" indexed="true" stored="false"/>
<field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false" />
<field name="text" type="string" indexed="true" stored="false" multiValued="true"/>
<uniqueKey>id</uniqueKey>
This is the code to dump the ids in the solr.
private void dumpSolr2() throws SolrServerException {
System.out.println("dumpSolr2");
SolrQuery query = new SolrQuery();
query.setQuery( "*:*" ); // get all
query.setFields("id"); // get the id field
query.setStart(0); // row to start on. First row is 0.
query.setRows(10); // number of rows to get at a time.
System.out.println(query.toString());
QueryResponse rsp = _solr.query( query );
List<BulkLoad1Data> bulkLoad1DataList = rsp.getBeans(BulkLoad1Data.class);
RplHelper1.rplSay("bulkLoad1DataList.size(): " + bulkLoad1DataList.size());
int count = 0;
Iterator<BulkLoad1Data> i = bulkLoad1DataList.iterator();
while (i.hasNext()) {
BulkLoad1Data bld = (BulkLoad1Data) i.next();
RplHelper1.rplSay("dumpSolr2 - count: " + ++count + " " +
" id: " + bld.getId());
}
}
I've written a QueryParser that is looks a little like Lucene's QueryParser. I've written a simple QParser and a QParserPlugin with which I can use my QueryParser.
One question that remains is: Where do I get a reference to the Analyzer configured in the Solr schema? Or how can I inject it to use the Analyzer into my query parser's constructor?
Here is my (simplified) solution in Java for using the configured Analyzer (and default field) from the schema file in MyQueryParser.
public class MyParserPlugin extends QParserPlugin {
#Override
public QParser createParser(String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest req) {
return new MyQParser(qstr, localParams, params, req);
}
}
class MyQParser extends QParser {
public MyQParser(String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest req) {
super(qstr, localParams, params, req);
}
#Override
public Query parse() throws ParseException {
// Getting info from the schema
String field = this.getReq().getSchema().getDefaultSearchFieldName();
Analyzer analyzer = this.getReq().getSchema().getQueryAnalyzer();
// Here we go
MyQueryParser parser = new MyQueryParser(Version.LUCENE_36, field, analyzer);
return parser.parse(this.qstr);
}
}