I have a large solr index, and I have noticed some fields are not updated correctly (the index is dynamic).
This has resulted in some fields having an empty "id" field.
I have tried these queries, but they didn't work:
id:''
id:NULL
id:null
id:""
id:
id:['' TO *]
Is there a way to query empty fields?
Thanks
Try this:
?q=-id:["" TO *]
One caveat! If you want to compose this via OR or AND you cannot use it in this form:
-myfield:*
but you must use
(*:* NOT myfield:*)
This form is perfectly composable. Apparently SOLR will expand the first form to the second, but only when it is a top node. Hope this saves you some time!
According to SolrQuerySyntax, you can use q=-id:[* TO *].
If you have a large index, you should use a default value
<field ... default="EMPTY" />
and then query for this default value.
This is much more efficient than q=-id:["" TO *]
You can also use it like this.
fq=!id:['' TO *]
If you are using SolrSharp, it does not support negative queries.
You need to change QueryParameter.cs (Create a new parameter)
private bool _negativeQuery = false;
public QueryParameter(string field, string value, ParameterJoin parameterJoin = ParameterJoin.AND, bool negativeQuery = false)
{
this._field = field;
this._value = value.Trim();
this._parameterJoin = parameterJoin;
this._negativeQuery = negativeQuery;
}
public bool NegativeQuery
{
get { return _negativeQuery; }
set { _negativeQuery = value; }
}
And in QueryParameterCollection.cs class, the ToString() override, looks if the Negative parameter is true
arQ[x] = (qp.NegativeQuery ? "-(" : "(") + qp.ToString() + ")" + (qp.Boost != 1 ? "^" + qp.Boost.ToString() : "");
When you call the parameter creator, if it's a negative value. Simple change the propertie
List<QueryParameter> QueryParameters = new List<QueryParameter>();
QueryParameters.Add(new QueryParameter("PartnerList", "[* TO *]", ParameterJoin.AND, true));
you can do it with filter query
q=*:*&fq=-id:*
A note added here, to make the field searchable first, it needs the field type in SOLR schema set to "indexed = true". Then you can use "field_name:*" for string type and "field_name:[* TO *]" for numeric type.
Related
I am very puzzled on the result returned by the Database.getQueryLocator().
global Database.QueryLocator start(Database.BatchableContext BC) {
string query = 'Select Label, Batch_Name__c, SObject_Name__c, External_system__c from Batch_Upload_Config__mdt where Is_Active__c = true';
System.debug(query);
return Database.getQueryLocator(query);
}
In the Query Editor, I entered the exact query and did get the correct data back. However when running the batch, instead of returning the records with 'Is_Active__c = true', it returns the records with 'Is_Active__c = false'.
Any pointers will be appreciated.
Looks like this is related to query on custom metadata types. When I changed to use custom object, I was able to get the correct result using Database.getQueryLocator().
I migrated my SQL data into AzureSearch document to try new search experience. I'm not able to filter data using .net sdk (3.0.4)
public IActionResult Search(string state, string category, string search, short pageNumber = 1, short pageSize = 10)
{
SearchIndexClient indexClient = new SearchIndexClient(searchServiceName, "search", new SearchCredentials(searchServiceApiKey));
DocumentSearchResult<SearchResultDto> results = null;
if (string.IsNullOrWhiteSpace(search))
search = "*";
if (state.Equals("All", StringComparison.InvariantCultureIgnoreCase))
state = string.Empty;
SearchParameters parameters = new SearchParameters()
{
Filter = "state eq " + state,
Top = pageSize,
Skip = (pageNumber - 1) * pageSize,
SearchMode = SearchMode.All,
IncludeTotalResultCount = true
};
try
{
results = indexClient.Documents.Search<SearchResultDto>(search, parameters);
return Ok(results.Results);
}
catch (Exception ex)
{
Console.WriteLine("Error querying index: {0}\r\n", ex.Message.ToString());
throw ex;
}
}
I'm getting error "Exception has been thrown by the target of an invocation."
parameters raw value : $count=true&$filter=state%20eq%20&queryType=simple&searchMode=all&$skip=0&$top=10
when I used parameters value in AzureSearch explore I'm getting error
Invalid expression: Expression expected at position 19 in 'state eq delhi eq '.\r\nParameter name: $filter
What is wrong with my code??
There are a few problems with your filter.
String literals in OData are delimited by single quotes. If you leave out the quotes, the string looks like a field name, but comparing fields to other fields is not allowed in Azure Search (also there is likely no field named delhi in your index). Try state eq 'delhi'.
The filter you tried with Search Explorer has an extra eq operator on the end: “state eq delhi eq “. If you remove the extra eq and put single quotes around delhi it should work.
Once you fix the syntax errors, the filter still might not work as intended. Filters are case-sensitive, so if the value you’re trying to match is actually ‘Delhi’ with a capital D, you won’t get a match. If the state field is matched to raw user input that might have the wrong case, it might be better to use the searchText parameter instead of Filter.
Consider this simple query which use full text searching on the Keywords field:
DECLARE #searchTerm VARCHAR(500) = 'painted'
SELECT * FROM StockCatalogueItems
WHERE (CONTAINS(KeyWords, #searchTerm))
This works as expected, but I need to do the same using a Dapper.net parameterised query. When using stored procedures, I create the full text parameter like this: "\"painted*\""
But using the same approach this doesn't work using dapper. No results are returned. This is the line in the query where I use the parameter:
AND (CONTAINS(KeyWords, #search))
and it's passed to the query like so:
return _context.Database.Connection.Query<StockProfileMatrix>(basequery, new
{
search = searchTerm
}
I can only assume that dapper is sanitising the string somehow, removing quotes perhaps?
Any ideas?
This works for me. However the tech stack am working on is .net core RTM and "Dapper": "1.50.0-rc3",
_dbConnection.QueryAsync<Guid>(#"select br.Id from Brand br where CONTAINS(br.Text,#Name)",new {Name = $"\"*{name}*\""}))
For completeness, I'll answer the question. The query syntax is correct, but the way in which the full-text parameter is created was obviously not. I created an extension method that formats the parameter:
public static string ToFullText(this string str)
{
string searchTerm = null;
if (!string.IsNullOrEmpty(str))
{
string[] keywords = str.Trim().Split(null);
foreach (var keyword in keywords)
{
searchTerm += string.Format("\"{0}*\" AND ", keyword);
}
if (searchTerm != null)
searchTerm = searchTerm.Substring(0, searchTerm.LastIndexOf(" AND "));
}
return searchTerm;
}
Then I call the method when I pass the parameter in to the dapper query:
_context.Database.Connection.Query<dynamic>(query, new
{
search = filter.SearchTerm.ToFullText()
});
I'm wondering how to order groups in a Solr result. I want to order the groups by numFound. I saw how to order the groups by score here, but that didn't seem to actually make a difference in the examples I looked at, and isn't exactly what I wanted.
In the xml you can see the number per group as numFound and that is what I want to sort the groups by, so for example I could see the largest group at the top.
<arr name="groups">
<lst>
<str name="groupValue">top secret</str>
<result name="doclist" numFound="12" start="0">
...
Any tips appreciated! Thanks!
This is an old question, but it is possible with two queries.
First query: bring back the field you're grouping by as a set of facets for your navigation state. You can limit the number of records returned to 0 here: you just need the facets. The number of facets you return should be the size of your page.
group_id:
23 (6)
143:(3)
5:(2)
Second query: Should be for the records, so no facets are required. The query should be an OR query for the facet field values returned from the first query. (group_id:23 OR group_id:143 OR group_id:5 and so on) and be grouped by the id you are used for grouping.
Sorting: reorder the records from query 2 to match the order from query 1.
That'll do it, with the proviso that I'm not sure how scalable that OR query will be. If you're looking to paginate, remember that you can offset facets: use that as the mechanism instead of offseting the records.
Sorting on the numFound is not possible as numFound is not an field in Solr.
Check the discussion mentioning it not being supported and I did not find a JIRA open for the issue as well.
Not possible since the last time I looked into this.
you can sort by using fields
consider an Example :
If you have 5 FACETS and COUNT associated with it.
Then you can sort by using the COUNTS of each fields.
It can be applicable to normal/non-facets fields .
public class FacetBean implements Category,Serializable {
private String facetName; //getter , setters
private long facetCount; // getter , setters
public FacetBean(String facetName, long count,) {
this.facetName = facetName;
this.count = count;
}}
Your calling method should be like this
private List<FacetBean> getFacetFieldsbyCount(QueryResponse queryResponse)
{
List<FacetField> flds = queryResponse.getFacetFields();
List<FacetBean> facetList = new ArrayList<FacetBean>();
FacetBean facet = null;
if (flds != null) {
for (FacetField fld : flds) {
facet = new FacetBean();
facet.setFacetName(fld.getName());
List<Count> counts = fld.getValues();
if (counts != null) {
for (Count count : counts) {
facet.setFacetCount(count.getCount());
}
}
facetList.add(facet);
}
}
Collections.sort(facetList,new Comparator<FacetBean>() {
public int compare(FacetBean obj1, FacetBean obj2) {
if(obj1.getFacetCount() > obj2.getFacetCount()) {
return (int)obj1.getFacetCount();
} else {
return (int)obj2.getFacetCount();
}
}
});
return facetList;
}
In The same URL They have mentioned something like.
sort -- >ex : For example, sort=popularity desc will cause the groups to be sorted according to the highest popularity doc
group.sort -- > you can apply your field here .
Hope it helps.
Using GAE-Java-JDO, is it possible to filter on the value of a specific element in a list?
WHAT WORKS
Normally, I would have the following:
#PersistenceCapable
class A {
String field1;
String field2;
// id, getters and setters
}
Then I would build a simple query:
Query q = pm.newQuery(A.class, "field1 == val");
q.declareParameters("String val");
List<A> list = new ArrayList<A>((List<A>) q.execute("foo"));
WHAT I WOULD LIKE
The above works fine. But what I would like to have is all of the fields stored in a list:
#PersistenceCapable
class AA {
ArrayList<String> fields;
// id, getters and setters
}
and then be able to query on a specific field in the list:
int index = 0;
Query q = pm.newQuery(A.class, "fields.get(index) == val");
q.declareParameters("int index, String val");
List<A> list = new ArrayList<A>((List<A>) q.execute(index, "foo"));
But this throws an exception:
org.datanucleus.store.appengine.query.DatastoreQuery$UnsupportedDatastoreFeatureException:
Problem with query
<SELECT FROM xxx.AA WHERE fields.get(index) == val PARAMETERS int index, String val,>:
Unsupported method <get> while parsing expression:
InvokeExpression{[PrimaryExpression{strings}].get(ParameterExpression{ui})}
My impression from reading the GAE-JDO doc is that this is not possible:
"The property value must be supplied by the application; it cannot refer to or be calculated in terms of other properties"
So... any ideas?
Thanks in advance!
If you only need to filter by index+value, then I think prefixing the actual list-values with their index should work. (If you need to also filter by actual values, then you'll need to store both lists.)
i.e. instead of the equivalent of
fields= ['foo', 'bar', 'baz] with query-filter fields[1] == 'bar'
you'd have
fields= ['0-foo', '1-bar', '2-baz'] with query-filter fields == '1-bar'
(but in java)