I've written a QueryParser that is looks a little like Lucene's QueryParser. I've written a simple QParser and a QParserPlugin with which I can use my QueryParser.
One question that remains is: Where do I get a reference to the Analyzer configured in the Solr schema? Or how can I inject it to use the Analyzer into my query parser's constructor?
Here is my (simplified) solution in Java for using the configured Analyzer (and default field) from the schema file in MyQueryParser.
public class MyParserPlugin extends QParserPlugin {
#Override
public QParser createParser(String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest req) {
return new MyQParser(qstr, localParams, params, req);
}
}
class MyQParser extends QParser {
public MyQParser(String qstr, SolrParams localParams, SolrParams params, SolrQueryRequest req) {
super(qstr, localParams, params, req);
}
#Override
public Query parse() throws ParseException {
// Getting info from the schema
String field = this.getReq().getSchema().getDefaultSearchFieldName();
Analyzer analyzer = this.getReq().getSchema().getQueryAnalyzer();
// Here we go
MyQueryParser parser = new MyQueryParser(Version.LUCENE_36, field, analyzer);
return parser.parse(this.qstr);
}
}
Related
As stated in the title I need to set a custom message key in KafkaSink. I cannot find any indication on how to achieve this in the Apache Flink 1.14 docs.
At the moment I'm correctly setting up the KafkaSink and the data payload is correctly written in the topic, but the key is null.
Any suggestions? Thanks in advance
You should implement a KafkaRecordSerializationSchema that sets the key on the ProducerRecord returned by its serialize method.
You'll create the sink more-or-less like this:
KafkaSink<UsageRecord> sink =
KafkaSink.<UsageRecord>builder()
.setBootstrapServers(brokers)
.setKafkaProducerConfig(kafkaProps)
.setRecordSerializer(new MyRecordSerializationSchema(topic))
.setDeliverGuarantee(DeliveryGuarantee.EXACTLY_ONCE)
.setTransactionalIdPrefix("my-record-producer")
.build();
and the serializer will be something like this:
public class MyRecordSerializationSchema implements
KafkaRecordSerializationSchema<T> {
private static final long serialVersionUID = 1L;
private String topic;
private static final ObjectMapper objectMapper =
JsonMapper.builder()
.build()
.registerModule(new JavaTimeModule())
.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false);
public MyRecordSerializationSchema() {}
public MyRecordSerializationSchema(String topic) {
this.topic = topic;
}
#Override
public ProducerRecord<byte[], byte[]> serialize(
T element, KafkaSinkContext context, Long timestamp) {
try {
return new ProducerRecord<>(
topic,
null, // choosing not to specify the partition
element.ts.toEpochMilli(),
element.getKey(),
objectMapper.writeValueAsBytes(element));
} catch (JsonProcessingException e) {
throw new IllegalArgumentException(
"Could not serialize record: " + element, e);
}
}
}
Note that this example is also setting the timestamp.
FWIW, this example comes from https://github.com/alpinegizmo/flink-mobile-data-usage/blob/main/src/main/java/com/ververica/flink/example/datausage/records/UsageRecordSerializationSchema.java.
This example is for scala programmers. Here, we are defining a key by generating UUID for each events.
import org.apache.flink.connector.kafka.sink.KafkaRecordSerializationSchema
import org.apache.kafka.clients.producer.ProducerRecord
import java.lang
class MyRecordSerializationSchema extends KafkaRecordSerializationSchema[String] {
override def serialize(element: String, context: KafkaRecordSerializationSchema.KafkaSinkContext, timestamp: lang.Long): ProducerRecord[Array[Byte], Array[Byte]] = {
return new ProducerRecord(
kafkaTopicName,
java.util.UUID.randomUUID.toString.getBytes,
element.getBytes
)
}
}
In the main class, will have to pass an instance of this class while defining the kafka sink like this:
val sinkKafka: KafkaSink[String] = KafkaSink.builder()
.setBootstrapServers(bootstrapServerUrl) //Bootstrap server url
.setRecordSerializer(new MyRecordSerializationSchema())
.build()
I am trying to add the params below (qf,bq) in a Solr query generated by Spring Data Solr.
Solr parameters are :
qf => Spring Data Solr Method?
bq => Spring Data Solr Method?
I was able to find the methods below
fq => addFilterQuery
fl => addProjectionOnField
defType => setDefType
qt => setRequestHandler
I saw an open issue qf https://jira.spring.io/browse/DATASOLR-153
How can i add the qf and bq params to Solr query built with Spring Data Solr.
Thanks
You can use the SolrCallback on template level to access the SolrClient and execute the query from there or register your own QueryParser for a custom query type.
Maybe something like:
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client);
template.registerQueryParser(EdismaxQuery.class, new EdisMaxQueryParser());
return template;
}
class EdismaxQuery extends SimpleQuery {
// ... add stuff you need. Maybe `autoRelax`
}
class EdisMaxQueryParser extends QueryParserBase<EdismaxQuery> {
DefaultQueryParser defaultQueryParser = new DefaultQueryParser();
#Override
public SolrQuery doConstructSolrQuery(EdismaxQuery source) {
// just use the default parser to construct the query string in first place.
SolrQuery target = defaultQueryParser.constructSolrQuery(source);
// add missing parameters
target.add("defType", "edismax");
target.add("qf", source....);
return target;
}
}
There're some changes in Spring Data Solr API 4.0, so you may need to change how you registered your own QueryParser a bit.
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client);
solrTemplate.registerQueryParser(EdismaxQuery.class, new EdisMaxQueryParser(new SimpleSolrMappingContext()));
return template;
}
public static class EdismaxQuery extends SimpleQuery {
private String defaultField;
private String minimumShouldMatch;
private String boostQuery;
private String queryField;
public EdismaxQuery(String queryString) {
super(queryString);
}
//... typical getter/setter
}
public static class EdisMaxQueryParser extends QueryParserBase<AbstractQueryDecorator> {
private final DefaultQueryParser defaultQueryParser;
public EdisMaxQueryParser(MappingContext<? extends SolrPersistentEntity<?>, SolrPersistentProperty> mappingContext) {
super(mappingContext);
defaultQueryParser = new DefaultQueryParser(mappingContext);
}
#Override
public SolrQuery doConstructSolrQuery(AbstractQueryDecorator queryDecorator, Class<?> domainType) {
// for some reason the API wrapped our query object with NamedObjectsQuery, so we need to unwrapped/get our actual query object first
EdismaxQuery query = (EdismaxQuery) queryDecorator.getDecoratedQuery();
// use defaultQueryParser to populate basic query parameters
SolrQuery solrQuery = defaultQueryParser.doConstructSolrQuery(query, domainType);
// set our own 'extra' parameter
if (query.getDefaultField() != null) {
solrQuery.add("df", query.getDefaultField());
}
if (query.getMinimumShouldMatch() != null) {
solrQuery.add("mm", query.getMinimumShouldMatch());
}
if (query.getQueryField() != null) {
solrQuery.add("qf", query.getQueryField());
}
if (query.getBoostQuery() != null) {
solrQuery.add("bq", query.getBoostQuery());
}
//...
return target;
}
}
Here's how you query with new EdismaxQuery object
EdismaxQuery query = new EdismaxQuery("hello world");
query.setDefType("edismax");
query.setRows(3);
query.setQueryField("text^2");
query.setMinimumShouldMatch("30%");
query.setBoostQuery("date:[NOW/DAY-1YEAR TO NOW/DAY]");
Page<ResultBean> results = solrTemplate.query("collection", query, ResultBean.class);
In order to avoid:
org.springframework.data.solr.core.QueryParserBase$NamedObjectsQuery
cannot be cast to EdismaxQuery
EdisMaxQueryParser should look like this:
class EdisMaxQueryParser extends QueryParserBase {
#Override
public SolrQuery doConstructSolrQuery(SolrDataQuery source) {
// your stuff
}
}
If you are going to add static qf expression to each select query it could be done in solrconfig.xml:
<requestHandler name="/select" class="solr.SearchHandler">
<lst name="defaults">
...
</lst>
<lst name="appends">
<str name="defType">edismax</str>
<str name="qf">offerId^100 vendorCode^100</str>
</lst>
...
</requestHandler>
you can use edismax library for spring-solr https://github.com/KmSYS/edismax-solr-spring
But to overcome the problem of overwrite the registered query parsers at afterPropertiesSet() you need to add following bean at config class,
#Bean
public SolrTemplate solrTemplate(SolrClient client) {
SolrTemplate template = new SolrTemplate(client) {
#Override
public void afterPropertiesSet() {
super.afterPropertiesSet();
registerQueryParser(SimpleEdismaxQuery.class, new EdisMaxQueryParser(new SimpleSolrMappingContext()));
}
};
template.afterPropertiesSet();
return template;
}
Also, sample code at https://github.com/KmSYS/edismax-solr-spring-sample
http://www.kmsys.tech/solr/edixmax-query-parser-template.html
So I have this class:
public class CustomValueSourceParser extends ValueSourceParser {
#Override
public ValueSource parse(FunctionQParser fqp) throws ParseException {
...
List<ValueSource> valSources = fqp.parseValueSourceList();
String iComeFromTheSolrFunctionArguments =
((LiteralValueSource)valSources.get(0)).getValue();
String iComeFromTheSolrQuery;
return new CustomValueSource(iComeFromTheSolrQuery, iComeFromTheSolrFunctionArguments);
}
}
I'd like to take the variable iComeFromTheSolrQuery from the solr query itself--not from the function arguments (because I will be calling the function multiple times and this string is very large).
Is there a way to do this? I tried adding a field to the search criteria, and then calling fqp.getParams(FIELD_NAME), but nothing came through.
Ideas?
Figured it out. What I wanted was to add a paramater. Not a field. When Formulating the query, I did this:
ModifiableSolrParams params = new ModifiableSolrParams();
params.set(PARAM_NAME_CONSTANT, paramValueString);
solrQuery.add(params);
Then in the above code I got the parameter like this:
public class CustomValueSourceParser extends ValueSourceParser {
#Override
public ValueSource parse(FunctionQParser fqp) throws ParseException {
...
List<ValueSource> valSources = fqp.parseValueSourceList();
String iComeFromTheSolrFunctionArguments =
((LiteralValueSource)valSources.get(0)).getValue();
String iComeFromTheSolrQuery=fqp.getParam(PARAM_NAME_CONSTANT);
return new CustomValueSource(iComeFromTheSolrQuery, iComeFromTheSolrFunctionArguments);
}
}
I have created a custom appender (will be used for Linux). For creating of this appender I used this article How write custom log4j appender
public class SolrAppender extends AppenderSkeleton {
private String path = null;
public void setPath(String path) { this.path = path; }
public String getPath() { return this.path; }
#Override
public boolean requiresLayout() {
return true;
}
#Override
public void close() {
}
#Override
public void activateOptions() {
super.activateOptions();
}
#Override
public synchronized void append(LoggingEvent event) {
SolrServer server = new HttpSolrServer(path);
SolrInputDocument document = new SolrInputDocument();
//some logic
UpdateResponse response = server.add(document);
server.commit();
}
Configuration of this appender is
# Solr appender
log4j.appender.SOLR = ricardo.solr.appender.QueryParser.SolrAppender
log4j.appender.SOLR.layout = org.apache.log4j.SimpleLayout
log4j.appender.SOLR.path = http://XX.XXX.XX.XX:8985/application/core
Appender works correct if path is hardcoded. Why path is not set via configuration?
From what I've seen so far, the name of a property of an appender, in the configuration, should start with an upper case character, so 'Path' instead of 'path', so you should use:
log4j.appender.SOLR.Path = http://XX.XXX.XX.XX:8985/application/core
Not sure why it's not the case for 'layout', though.
I want to know if it is efficient in JSF to define EntityManager and TypedQuery in a bundle class that is supposed to read messages from database?
What if I create an instance of a #Stateless bean and use its functions that return query results inside the bundle class?
UPDATE: Included some code:
protected class DBControl extends Control{
#Override
public ResourceBundle newBundle
(String baseName, Locale locale, String format, ClassLoader loader, boolean reload)
throws IllegalAccessException, InstantiationException, IOException
{
return new ArticleResources(locale);
}
protected class ArticleResources extends ListResourceBundle{
private Locale locale;
public ArticleResources (Locale locale){
this.locale = locale;
}
String language = locale.getLanguage();
#Override
protected Object[][] getContents(){
TypedQuery<ArticleLcl> query = em.createNamedQuery("ArticleLcl.findForLocale", ArticleLcl.class);
query.setParameter("lang", language);
List<ArticleLcl> articles = query.getResultList();
Object[][] allArticles = new Object[articles.size()][3];
int i = 0;
for(Iterator<ArticleLcl> it = articles.iterator(); it.hasNext();){
ArticleLcl article = it.next();
allArticles[i] = new Object[]{article.getArticleId().getArticleId().toString(),article.getArticleTitle()};
messages.put(article.getArticleId().getArticleId().toString(),article.getArticleTitle());
i++;
}
return allArticles;
}
}
By the way this code does not work and my entity manager is null. But I wonder am I doing the right thing?