I am new to solr and this is my first attempt at indexing solr data, I am getting the following exception while indexing,
org.apache.solr.common.SolrException: Invalid Date String:'2011-01-07'
at org.apache.solr.schema.DateField.parseMath(DateField.java:165)
at org.apache.solr.schema.TrieDateField.createField(TrieDateField.java:169)
at org.apache.solr.schema.SchemaField.createField(SchemaField.java:98)
at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:204)
at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:277)
I understand from reading some articles that Solr stores time only in UTC, this is the query i am trying to index,
Select id,text,'language',links,tweetType,source,location, bio,url,utcOffset,timeZone,frenCnt,createdAt,createdOnGMT,createdOnServerTime,follCnt,favCnt,totStatusCnt,usrCrtDate,humanSentiment,replied,replyMsg,classified,locationDetail, geonameid,country,continent,placeLongitude,placeLatitude,listedCnt,hashtag,mentions,senderInfScr, createdOnGMTDate,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+05:30'),'%Y-%m-%d') as IST,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+01:00'),'%Y-%m-%d') as ECT,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+02:00'),'%Y-%m-%d') as EET,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+03:30'),'%Y-%m-%d') as MET,sign(classified) as sentiment from
Why i am doing this timezone conversion is because i need to group results by the user timezone. How can i achieve this?
Regards,
Rohit
Solr dates must be in the form 1995-12-31T23:59:59Z. You're only giving the date part, but not the time.
See the DateField javadocs for more details.
Date faceting is entirely driven by query params, so if we index your events using the "true" time that they happend at (formatted as a string in UTC) you can then select your date ranges using whatever timezone offset is specified by your user at query time as a UTC offset.
facet.range = dateField
facet.range.start = 2011-01-01T00:00:00Z+${useroffset}MINUTES
facet.range.gap = +1DAY
This would return result in the users timezone and there is actually no need to timezone conversion the query and indexing that column separately.
Regards,
Rohit
Credit For Answer: Chris Hostetter (Solr User Group )
Related
I have some documents inserted in vespa. All the documents have a particular timestamp(Some has 1:00 PM, some has 1:15pm etc). If I query to vespa with current timestamp and timestamp after 15 minutes interval(let it be 12:55 PM(current time) and 1:10PM in my query), then vespa must return the document with the timestamp(ie 1:00PM ) lies between them. How can I achieve it? Please help
See https://docs.vespa.ai/documentation/query-language.html and the range operator which works on numeric fields.
Could you please guide me on how to write a gremlin query that will return only projects that have started past a specific date?
My first query returns all project vertices within program05:
g.V('program05').has('partitionkey', 'program05').out('hasprojects')
I would like to filter it down to return only projects that have started after '20/19/2018 4:37:12 PM' , a program vertex has a propertt of startDate
I have tried :
g.V('program05').has('partitionkey', 'program05').out('hasprojects').has('startDate').has('startDate',gt, '20/19/2018 4:37:12 PM')
but i get an Error: Unable to resolve symbol 'lt' in the current context. I have tried other opions too with no luck
The predicate logic (javadoc) for strings seems to work based on ASCII values of the string, so your current storage of the date in text format will not work month over month.
I would suggest storing that as epoch seconds and then using the following query to get that data you want.
Assuming you meant the date 2/19/2018 4:37:12 PM
g.V()
.has('partitionkey','program05')
.out('hasprojects')
.has('startDate',P.gt(1519058232))
I am running a SOQL query and getting a date/time field value as per user's timezone. I have a requirement to change the timezone in EST in the SOQL query itself. For example: User is in PST and while running SOQL query I am getting createdDate field value in PST. But I need that the result should give the createdDate value in EST.
Date in PST = SELECT CreatedDate FROM Opportunity
Date in EST = SELECT ????? FROM Opportunity
Thanks.
From Date Formats and Date Literals
dateTime field values are stored as Coordinated Universal Time (UTC). When a dateTime value is returned in Salesforce, it’s adjusted for the time zone specified in your org preferences. SOQL queries, however, return dateTime field values as UTC values. If you want to process these values in different time zones, your application might need to handle the conversion.
You can use format(dateFormatString, timezone) method of Datetime class or dateGMT() to convert date in Apex.
If you need to display date somewhere outside of Salesforce, you will need to do this on your end.
I was looking to implement SOLR for a Hotel bookings site. Search based on location, hotel names, facilities works very well and so does the faceting. What I have not been able to figure out is how to search for a hotel given Checkin and Checkout dates.
Eg: User will search for search query - "Hotels in Newyork" and select CheckIn Date: 10th Feb 2012 and CheckOut Date: 12 Feb 2012 from the date selection box.
This is how I have the data -
Hotel_Name 10thFeb2012 11thFEB2012 ........ 31DEC2012
Hotel1 2room 3room 10rooms
Hotel2 1room 4room ........ 12rooms
Now if the query is for Hotel2 for 3rooms from checkin Date 10thFeb2012 to 11thFeb2012 it shdnt match because there is only one room available for 10thFeb.
IF the query for Hotel2 is for 1 room from checkin Date 10thFeb2012 to 11thFeb2012 then it should be part of search result.
Use the ISO 8601 format for your date-times.
Complete date plus hours, minutes, seconds and a decimal fraction of a second
YYYY-MM-DDThh:mm:ss.sTZD (eg 1997-07-16T19:20:30.45Z)
Both your database and Solr will understand date-times from strings that conform to that format.
So,
store the data in DB and Solr with compatible date-time formats. (On the back of my head, Solr must have a Z appended to the date-time, else its invalid).
your search interface must format all dates in that format to query solr.
Solr can do conditional expressions, facets, range bucket faceting etc with dates.
I would go with the following schema:
hotel_name : string (for faceting)
hotel_name_searchable : text (for searching, this is a copy field:look it up)
room_id : string
start_date : date (when the room is availabe)
end_date : date (if not booked, set it to an infinite date, say 2040)
For each room you are ever tracking, store the date-times between which it is free.
You can search for rooms between the start_date and end_date.
Do faceting on hotel_name so your search for rooms "checkin Date 10thFeb2012 to 11thFeb2012" gets you:
Hotel1:[r1,r2,r3]
Hotel2:[r8]
Hotel3:[r2,r3,r4]
Faceting on hotel_name filters to one hotel, facet.mincount on room_id can return hotels having the required number of rooms.
A little warning: I may be a bit rusty on faceting, as I used to do a lot of processing on Solr results itself.
I have got a list of resume ID's with the date field and the experience in some range
My queries are working with standard query handler but not in dismax.
How can I check for the date ranges in between suppose 2009-02-02 to 2010-01-01 using solr's dismax query handler
And HOW to configure range queries like minimum experience [3 TO 5] in dismax..
It works with standard handler..but with qt=dismax, it doesnt work..
Have you tried adding filter queries so the it looks something like this:
fq=date:[2009-02-02T23:59:59.999Z TO 2010-01-01T23:59:59.999Z&fq=experience:[3 TO 5]
solr stores date type field in UTC format, so while querying it needs the date criteria to be in UTC format like date:[2009-02-02T00:00:00.0Z TO 2010-01-01T00:00:00.0Z]