I am trying to add the params below (qf,bq) in a Solr query generated by Spring Data Solr.
Solr parameters are :
qf => Spring Data Solr Method?
bq => Spring Data Solr Method?
I was able to find the methods below
fq => addFilterQuery
fl => addProjectionOnField
defType => setDefType
qt => setRequestHandler
I saw an open issue qf https://jira.spring.io/browse/DATASOLR-153
How can i add the qf and bq params to Solr query built with Spring Data Solr.
Thanks
You can use the SolrCallback on template level to access the SolrClient and execute the query from there or register your own QueryParser for a custom query type.
Maybe something like:
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client);
template.registerQueryParser(EdismaxQuery.class, new EdisMaxQueryParser());
return template;
}
class EdismaxQuery extends SimpleQuery {
// ... add stuff you need. Maybe `autoRelax`
}
class EdisMaxQueryParser extends QueryParserBase<EdismaxQuery> {
DefaultQueryParser defaultQueryParser = new DefaultQueryParser();
#Override
public SolrQuery doConstructSolrQuery(EdismaxQuery source) {
// just use the default parser to construct the query string in first place.
SolrQuery target = defaultQueryParser.constructSolrQuery(source);
// add missing parameters
target.add("defType", "edismax");
target.add("qf", source....);
return target;
}
}
There're some changes in Spring Data Solr API 4.0, so you may need to change how you registered your own QueryParser a bit.
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client);
solrTemplate.registerQueryParser(EdismaxQuery.class, new EdisMaxQueryParser(new SimpleSolrMappingContext()));
return template;
}
public static class EdismaxQuery extends SimpleQuery {
private String defaultField;
private String minimumShouldMatch;
private String boostQuery;
private String queryField;
public EdismaxQuery(String queryString) {
super(queryString);
}
//... typical getter/setter
}
public static class EdisMaxQueryParser extends QueryParserBase<AbstractQueryDecorator> {
private final DefaultQueryParser defaultQueryParser;
public EdisMaxQueryParser(MappingContext<? extends SolrPersistentEntity<?>, SolrPersistentProperty> mappingContext) {
super(mappingContext);
defaultQueryParser = new DefaultQueryParser(mappingContext);
}
#Override
public SolrQuery doConstructSolrQuery(AbstractQueryDecorator queryDecorator, Class<?> domainType) {
// for some reason the API wrapped our query object with NamedObjectsQuery, so we need to unwrapped/get our actual query object first
EdismaxQuery query = (EdismaxQuery) queryDecorator.getDecoratedQuery();
// use defaultQueryParser to populate basic query parameters
SolrQuery solrQuery = defaultQueryParser.doConstructSolrQuery(query, domainType);
// set our own 'extra' parameter
if (query.getDefaultField() != null) {
solrQuery.add("df", query.getDefaultField());
}
if (query.getMinimumShouldMatch() != null) {
solrQuery.add("mm", query.getMinimumShouldMatch());
}
if (query.getQueryField() != null) {
solrQuery.add("qf", query.getQueryField());
}
if (query.getBoostQuery() != null) {
solrQuery.add("bq", query.getBoostQuery());
}
//...
return target;
}
}
Here's how you query with new EdismaxQuery object
EdismaxQuery query = new EdismaxQuery("hello world");
query.setDefType("edismax");
query.setRows(3);
query.setQueryField("text^2");
query.setMinimumShouldMatch("30%");
query.setBoostQuery("date:[NOW/DAY-1YEAR TO NOW/DAY]");
Page<ResultBean> results = solrTemplate.query("collection", query, ResultBean.class);
In order to avoid:
org.springframework.data.solr.core.QueryParserBase$NamedObjectsQuery
cannot be cast to EdismaxQuery
EdisMaxQueryParser should look like this:
class EdisMaxQueryParser extends QueryParserBase {
#Override
public SolrQuery doConstructSolrQuery(SolrDataQuery source) {
// your stuff
}
}
If you are going to add static qf expression to each select query it could be done in solrconfig.xml:
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
...
</lst>
<lst name="appends">
<str name="defType">edismax</str>
<str name="qf">offerId^100 vendorCode^100</str>
</lst>
...
</requestHandler>
you can use edismax library for spring-solr https://github.com/KmSYS/edismax-solr-spring
But to overcome the problem of overwrite the registered query parsers at afterPropertiesSet() you need to add following bean at config class,
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client) {
#Override
public void afterPropertiesSet() {
super.afterPropertiesSet();
registerQueryParser(SimpleEdismaxQuery.class, new EdisMaxQueryParser(new SimpleSolrMappingContext()));
}
};
template.afterPropertiesSet();
return template;
}
Also, sample code at https://github.com/KmSYS/edismax-solr-spring-sample
http://www.kmsys.tech/solr/edixmax-query-parser-template.html
Related
I am using spring boot, apache cxf, jaxrs and hibernate validator . I am trying to create custom constraint for method parameter. Reason is that #Valid does not have groups.How do I register or add this to hibernate validator so that it behaves like #Valid ie ExecutableValidator. below is the code
#Target({ElementType.PARAMETER})
#Retention(RetentionPolicy.RUNTIME)
#Constraint(validatedBy=MyValidator.class)
#Documented
public #interface ValidInput {
String message() default "com.xyz.message";
Class<?>[] groups() default { };
Class<? extends Payload>[] payload() default { };
}
#Provider
#Component
public class MyValidator implements ConstraintValidator<ValidInput, Entity> {
private static Validator validator;
private Class<?>[] groups ;
Class<? extends Payload>[] payload;
static{
ValidatorFactory factory = Validation.buildDefaultValidatorFactory();
validator = factory.getValidator();
}
#Override
public void initialize(ValidInput constraintAnnotation) {
this.groups = constraintAnnotation.groups();
this.payload = constraintAnnotation.payload();
}
#Override
public boolean isValid(Entity value, ConstraintValidatorContext context) {
//logic here
}
}
now, when I use the constraint on a rest service method parameter, the validator is not at all called . Instead of #Valid , I want to use #ValidInput.
pls help
Hope you have Enabled Validation feature.
#Bean
public JAXRSBeanValidationFeature beanValidationFeature(){
JAXRSBeanValidationFeature feature = new JAXRSBeanValidationFeature();
return feature;
}
#Bean
public ValidationExceptionMapper validationExceptionMapper(){
return new ValidationExceptionMapper();
}
Faceting in solr work on the result returned by query, but I want it to be done on top of documents which are returned after limiting rows to some value.For example if q return 1500 document and I am taking first 1000 rows, I want faceting to be applied on 1000 instead of 1500.I am writing a custom search component to generate facets. Following is the implementation:
import java.io.IOException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.TreeMap;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.IndexableField;
import org.apache.solr.handler.component.ResponseBuilder;
import org.apache.solr.handler.component.SearchComponent;
import org.apache.solr.request.SolrQueryRequest;
import org.apache.solr.search.DocIterator;
import org.apache.solr.search.DocList;
import org.apache.solr.search.SolrIndexSearcher;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.biginfolabs.subfacet.MapSorter;
public class SubFacetComponent extends SearchComponent {
private static Logger logger = LoggerFactory
.getLogger(SubFacetComponent.class);
private SolrQueryRequest req = null;
private long numDoc = 0;
// private static final String field = "annotations";
#Override
public String getDescription() {
// TODO Auto-generated method stub
return null;
}
#Override
public void prepare(ResponseBuilder arg0) throws IOException {
// TODO Auto-generated method stub
}
#Override
public void process(ResponseBuilder rb) throws IOException {
// TODO Auto-generated method stub
long startTime = System.currentTimeMillis();
req = rb.req;
String[] fields = {"annotations"};
// req.getParams().getParams("facet.field");
logger.info("Facet component");
if (rb.getResults() != null) {
DocList docs = rb.getResults().docList;
numDoc = docs.size();
for (String field : fields) {
DocIterator docItr = docs.iterator();
SolrIndexSearcher searcher = req.getSearcher();
Map<String, List<String>> map = new HashMap<String, List<String>>();
List<String> list = new ArrayList<String>();
while (docItr.hasNext()) {
int docId = docItr.nextDoc();
Document doc = searcher.doc(docId);
list.add(doc.get("id"));
Set<String> keySet = new HashSet<String>();
if (doc.get(field) != null) {
IndexableField[] indexableFields = doc.getFields(field);
for (IndexableField indexableField : indexableFields) {
String str = indexableField.stringValue();
String[] pathVariableArray = str.split("/");
String key = "";
String separator = "";
for (String split : pathVariableArray) {
key += separator + split;
keySet.add(key);
separator = "/";
}
}
for (String str : keySet) {
List<String> arrayList = new ArrayList<String>();
if (map.containsKey(str)) {
arrayList = map.get(str);
}
arrayList.add(doc.getField("_version_").toString());
map.put(str, arrayList);
}
}
}
System.out.println(list);
Map<String, Integer> finalMap = new HashMap<String, Integer>();
for (String key : map.keySet()) {
if (map.containsKey(key)) {
finalMap.put(key, map.get(key).size());
System.out.println(key + ": " + map.get(key).size());
}
}
Map<String, TreeMap<String, Integer>> facetMap = new HashMap<String, TreeMap<String, Integer>>();
TreeMap<String, Integer> sortedMap = new TreeMap(new MapSorter(
finalMap));
sortedMap.putAll(finalMap);
rb.rsp.add("subfacet", sortedMap);
}
} else {
logger.warn("You must specify 'subfacet' params in solr query !!");
}
long enTime = System.currentTimeMillis();
logger.info("Time taken to generate facets for " + numDoc
+ " documents is " + (enTime - startTime) + " ms");
}
}
I am trying to add the response using rb.rsp.add("subfacet", sortedMap);, which seems to be setting the subfacet in rsp object but the response returned to solr UI doesn't contain this object.What am I missing here ?
Following is my select request handler:
<requestHandler name="/select" class="solr.SearchHandler">
<!-- default values for query parameters can be specified, these
will be overridden by parameters in the request
-->
<lst name="defaults">
<str name="echoParams">explicit</str>
<int name="rows">10</int>
</lst>
<arr name="last-components">
<str>subfacetcomponent</str>
</arr>
</requestHandler>
Edit: It works fine if solr is used as single node and single shard but not on cloud mode.
fixed it by overriding and writing logic in public void finishStage(ResponseBuilder rb) instead of process().
So I have this class:
public class CustomValueSourceParser extends ValueSourceParser {
#Override
public ValueSource parse(FunctionQParser fqp) throws ParseException {
...
List<ValueSource> valSources = fqp.parseValueSourceList();
String iComeFromTheSolrFunctionArguments =
((LiteralValueSource)valSources.get(0)).getValue();
String iComeFromTheSolrQuery;
return new CustomValueSource(iComeFromTheSolrQuery, iComeFromTheSolrFunctionArguments);
}
}
I'd like to take the variable iComeFromTheSolrQuery from the solr query itself--not from the function arguments (because I will be calling the function multiple times and this string is very large).
Is there a way to do this? I tried adding a field to the search criteria, and then calling fqp.getParams(FIELD_NAME), but nothing came through.
Ideas?
Figured it out. What I wanted was to add a paramater. Not a field. When Formulating the query, I did this:
ModifiableSolrParams params = new ModifiableSolrParams();
params.set(PARAM_NAME_CONSTANT, paramValueString);
solrQuery.add(params);
Then in the above code I got the parameter like this:
public class CustomValueSourceParser extends ValueSourceParser {
#Override
public ValueSource parse(FunctionQParser fqp) throws ParseException {
...
List<ValueSource> valSources = fqp.parseValueSourceList();
String iComeFromTheSolrFunctionArguments =
((LiteralValueSource)valSources.get(0)).getValue();
String iComeFromTheSolrQuery=fqp.getParam(PARAM_NAME_CONSTANT);
return new CustomValueSource(iComeFromTheSolrQuery, iComeFromTheSolrFunctionArguments);
}
}
I have created a custom appender (will be used for Linux). For creating of this appender I used this article How write custom log4j appender
public class SolrAppender extends AppenderSkeleton {
private String path = null;
public void setPath(String path) { this.path = path; }
public String getPath() { return this.path; }
#Override
public boolean requiresLayout() {
return true;
}
#Override
public void close() {
}
#Override
public void activateOptions() {
super.activateOptions();
}
#Override
public synchronized void append(LoggingEvent event) {
SolrServer server = new HttpSolrServer(path);
SolrInputDocument document = new SolrInputDocument();
//some logic
UpdateResponse response = server.add(document);
server.commit();
}
Configuration of this appender is
# Solr appender
log4j.appender.SOLR = ricardo.solr.appender.QueryParser.SolrAppender
log4j.appender.SOLR.layout = org.apache.log4j.SimpleLayout
log4j.appender.SOLR.path = http://XX.XXX.XX.XX:8985/application/core
Appender works correct if path is hardcoded. Why path is not set via configuration?
From what I've seen so far, the name of a property of an appender, in the configuration, should start with an upper case character, so 'Path' instead of 'path', so you should use:
log4j.appender.SOLR.Path = http://XX.XXX.XX.XX:8985/application/core
Not sure why it's not the case for 'layout', though.
I've written a QueryParser that is looks a little like Lucene's QueryParser. I've written a simple QParser and a QParserPlugin with which I can use my QueryParser.
One question that remains is: Where do I get a reference to the Analyzer configured in the Solr schema? Or how can I inject it to use the Analyzer into my query parser's constructor?
Here is my (simplified) solution in Java for using the configured Analyzer (and default field) from the schema file in MyQueryParser.
public class MyParserPlugin extends QParserPlugin {
#Override
public QParser createParser(String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest req) {
return new MyQParser(qstr, localParams, params, req);
}
}
class MyQParser extends QParser {
public MyQParser(String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest req) {
super(qstr, localParams, params, req);
}
#Override
public Query parse() throws ParseException {
// Getting info from the schema
String field = this.getReq().getSchema().getDefaultSearchFieldName();
Analyzer analyzer = this.getReq().getSchema().getQueryAnalyzer();
// Here we go
MyQueryParser parser = new MyQueryParser(Version.LUCENE_36, field, analyzer);
return parser.parse(this.qstr);
}
}