I'm running a Solr json facet query to get a range of dates for a field (updated_at).
{"updated_at":{
"type":"range",
"field":"updated_at",
"start":"1970-01-01T00:00:00Z",
"end":"NOW",
"gap":"%2B1YEAR",
"missing":false,
"limit":-1}
}'
This works, and shows me yearly ranges starting at 1970, but my earliest date right now is 2019.
How could I dynamically set the start property to the min(updated_at)?
I've tried "start":"min(updated_at)" but that gives me an error:
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","org.apache.solr.common.SolrException"],
"msg":"Can't parse value min(updated_at) for field: updated_at",
"code":400}
I am using solr 4.2 and I have a date field in the database which converted erroneously in solr's date field e.g. the value 2021-05-10 which is stored in the table, is converted into the value 2021-05-09T22:00:00Z in solr. So, what have to do in order to be able to handle the date values as they are, without the time/timezone part?
I'm having my SOLR index as:
{'year':2002
'user_entries':['user1']},
{'year':2003
'user_entries':['user2']},
{'year':2002
'user_entries':['user1']},
Expected result
{facet_fields:{2002:{'user1':2}, 2003:{'user2':1}}}
I can use fq=year:2002 to extract the facets on user_entries to extract the count of entries of each user in year 2002 as
/solr/rss/select?q=:&fq=year:2002&wt=json&indent=true&facet=true&facet.query=year:2002&facet.field=user_entries
But I want to extract the user_entries of each year from: 2002 to 2010 individually without summing them all up.
Current Approach:
1. solr/rss/select?q=*:*&fq=year:2002&wt=json&indent=true&facet=true&facet.query=year:2002&facet.field=user_entries
2. solr/rss/select?q=*:*&fq=year:2003&wt=json&indent=true&facet=true&facet.query=year:2003&facet.field=user_entries...
So can I have a single multiple filter query instead of multiple queries for each year with which I can extract individual year data without summing them up as with default OR operation?
If you are using Solr 4.0 or newer then you can use pivot facets:
?q=*:*
&facet=true
&facet.pivot=year,user_entries
&f.user_entries.facet.limit=3
But the response is not exactly the same as regular field facets, so you may have to change your parsing code. You will get the count for the "2002" value of the year field, and for each combination of "2002" and values of the user_entries fields (limited to 3), followed by the same for the "2003" value.
I was looking to implement SOLR for a Hotel bookings site. Search based on location, hotel names, facilities works very well and so does the faceting. What I have not been able to figure out is how to search for a hotel given Checkin and Checkout dates.
Eg: User will search for search query - "Hotels in Newyork" and select CheckIn Date: 10th Feb 2012 and CheckOut Date: 12 Feb 2012 from the date selection box.
This is how I have the data -
Hotel_Name 10thFeb2012 11thFEB2012 ........ 31DEC2012
Hotel1 2room 3room 10rooms
Hotel2 1room 4room ........ 12rooms
Now if the query is for Hotel2 for 3rooms from checkin Date 10thFeb2012 to 11thFeb2012 it shdnt match because there is only one room available for 10thFeb.
IF the query for Hotel2 is for 1 room from checkin Date 10thFeb2012 to 11thFeb2012 then it should be part of search result.
Use the ISO 8601 format for your date-times.
Complete date plus hours, minutes, seconds and a decimal fraction of a second
YYYY-MM-DDThh:mm:ss.sTZD (eg 1997-07-16T19:20:30.45Z)
Both your database and Solr will understand date-times from strings that conform to that format.
So,
store the data in DB and Solr with compatible date-time formats. (On the back of my head, Solr must have a Z appended to the date-time, else its invalid).
your search interface must format all dates in that format to query solr.
Solr can do conditional expressions, facets, range bucket faceting etc with dates.
I would go with the following schema:
hotel_name : string (for faceting)
hotel_name_searchable : text (for searching, this is a copy field:look it up)
room_id : string
start_date : date (when the room is availabe)
end_date : date (if not booked, set it to an infinite date, say 2040)
For each room you are ever tracking, store the date-times between which it is free.
You can search for rooms between the start_date and end_date.
Do faceting on hotel_name so your search for rooms "checkin Date 10thFeb2012 to 11thFeb2012" gets you:
Hotel1:[r1,r2,r3]
Hotel2:[r8]
Hotel3:[r2,r3,r4]
Faceting on hotel_name filters to one hotel, facet.mincount on room_id can return hotels having the required number of rooms.
A little warning: I may be a bit rusty on faceting, as I used to do a lot of processing on Solr results itself.
I am new to solr and this is my first attempt at indexing solr data, I am getting the following exception while indexing,
org.apache.solr.common.SolrException: Invalid Date String:'2011-01-07'
at org.apache.solr.schema.DateField.parseMath(DateField.java:165)
at org.apache.solr.schema.TrieDateField.createField(TrieDateField.java:169)
at org.apache.solr.schema.SchemaField.createField(SchemaField.java:98)
at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:204)
at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:277)
I understand from reading some articles that Solr stores time only in UTC, this is the query i am trying to index,
Select id,text,'language',links,tweetType,source,location, bio,url,utcOffset,timeZone,frenCnt,createdAt,createdOnGMT,createdOnServerTime,follCnt,favCnt,totStatusCnt,usrCrtDate,humanSentiment,replied,replyMsg,classified,locationDetail, geonameid,country,continent,placeLongitude,placeLatitude,listedCnt,hashtag,mentions,senderInfScr, createdOnGMTDate,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+05:30'),'%Y-%m-%d') as IST,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+01:00'),'%Y-%m-%d') as ECT,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+02:00'),'%Y-%m-%d') as EET,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+03:30'),'%Y-%m-%d') as MET,sign(classified) as sentiment from
Why i am doing this timezone conversion is because i need to group results by the user timezone. How can i achieve this?
Regards,
Rohit
Solr dates must be in the form 1995-12-31T23:59:59Z. You're only giving the date part, but not the time.
See the DateField javadocs for more details.
Date faceting is entirely driven by query params, so if we index your events using the "true" time that they happend at (formatted as a string in UTC) you can then select your date ranges using whatever timezone offset is specified by your user at query time as a UTC offset.
facet.range = dateField
facet.range.start = 2011-01-01T00:00:00Z+${useroffset}MINUTES
facet.range.gap = +1DAY
This would return result in the users timezone and there is actually no need to timezone conversion the query and indexing that column separately.
Regards,
Rohit
Credit For Answer: Chris Hostetter (Solr User Group )