We are using Solr 9.1. We want to calculate a monthly time series metric. We are using following request
https://localhost/solr/metrics/stream?expr=timeseries(
metrics,
q="type_i:4",
field="ended_at_dt",
start="NOW/MONTH",
end="NOW/MONTH+5MONTH",
gap="+1MONTH"
)&TZ=Asia/Kolkata
Irrespective of TZ parameter we are getting monthly results based on UTC tie zone shown in below screnshot
While in SOLR documentation It is mentioned
By default, all date math expressions are evaluated relative to the UTC TimeZone, but the TZ parameter can be specified to override this behaviour, by forcing all date based addition and rounding to be relative to the specified time zone.
For example, the following request will use range faceting to facet over the current month, "per day" relative UTC:
I have tested below example given in document
http://localhost/solr/metrics/select?q=*:*&facet.range=ended_at_dt&facet=true&facet.range.start=NOW/MONTH&facet.range.end=NOW/MONTH+5MONTH&facet.range.gap=+1MONTH&TZ=Asia/Kolkata
And it is giving result as expected as shown in below screenshot
Please guide how to use timeseries expression to show results based on timezone.
Related
I have a table with column "date" in YYYY-MM-DD format HH:MM:SS:MMM (2015-01-27 11:22:03:742). I'm trying to make a time series with the dimension of month/year grouping, to display the total number of records by period.
Settings:
period dimension: date (type: date and time)
period: date (type: year and month)
metric: record count
My time graph doesn't display anything. Can someone help me identify what's going on?
formatDate is the column created with the expression:
PARSE_DATETIME("%Y-%m-%d %H:%M:%S",REGEXP_EXTRACT( create_date,"(.*):[0-9]*"))
Using the date in its standard format, as mentioned at the beginning of the question, the same happens.
When entering dates (original and formatted), both appear with null values.
The milliseconds have to be separated by a . not a :. An option is to import your date a as string/text and add a calculated field, which parse the string in Data Studio:
PARSE_DATETIME("%Y-%m-%d %H:%M:%S",REGEXP_EXTRACT( data_field,"(.*):[0-9]*"))
If the dates are several years in the past, please adjust the Default date range in your graph:
I leave the solution to my problem to the community.
The problem is in the date format. Failed to get Google Data Studio to receive a date with milliseconds. By removing the milliseconds it was possible to work with the dates normally, managing to apply the available functions.
Note: It may be a knowledge limitation, but none of the date formatting functions work if the datetime field contains milliseconds (FORMAT_DATETIME, PARSE_DATETIME,...)
I have a data source consisting of date, URL, and metric (number) which changes for each URL from date to date.
So I built a report with this setup:
date as Date Range Dimension,
URL as Dimension,
metric (average) as Metric,
default date range as today,
comparison between date and previous time period.
With this setup, I got the metric for today for each URL and the comparison between today and yesterday. This works like a charm.
Then I wanted to see the average metric for today in comparison to yesterday and changed the setup to this:
date as Date Range Dimension,
date as Dimension,
metric (average) as Metric (to see the average metric for today and comparison of it with yesterday)
Now I see the average metric's value for today, as it should - but no comparison value.
What am I doing wrong? How do I display the comparison of metric between then date and the previous time period?
EDIT 1: Here is report link with editing rights. And here is the data source - Google Sheets document.
On the screenshot you see:
in the top table every URL with its today's metric and comparison to yesterday,
in the bottom table you see the date with the average metric for today - but no comparison value.
Edit 2: I added to the data source (Google Sheets) an additional column where the date is 100% correctly recognized by Google Sheets as date (not as DateTime, as in original column). But no success here too.
I linked Firebase to BigQuery and start using Google Data Studio to create a table to list users by "User Pseudo ID".
My goal is to calculate the difference between two dates, the date of first_open and the date of app_remove to come up with an average retention time.
How can I write the right query in Data Studio?
It can be achieved using the three step process below:
1) HH:MM:SS
The Calculated Field below uses the DATETIME_DIFF function to find the difference between app_remove and first_open, and displays the difference in SECOND (for future reference, set the third input DATETIME_DIFF as required, for example, to view the difference in days, set the input to DAY):
DATETIME_DIFF(app_remove, first_open, SECOND)
2) Type (HH:MM:SS)
Number > Duration (Sec.)
3) Aggregation (HH:MM:SS)
AVG
Google Data Studio Report and a GIF to elaborate:
DATE_DIFF may be what you are looking for.
That is if first_open and app_remove are date fields or date expressions
I can't find a way of getting a date from Azure Search using only the date.
The dates in the index are like: "2019-10-15T18:00:00Z","2019-10-22T18:00:00Z","2019-10-29T18:00:00Z"
If I try StartDate/any(s: s eq 2019-10-15T18:00:00Z) I get results
But with StartDate/any(s: s eq 2019-10-15) nothing comes up
I have tried usin the date OData function like so: StartDate/any(s: date(s) eq 2019-10-15) but I get an error 'Function 'date' is not supported'.
Is there any way to get dates without using the time part?
The use of date literals in filters in Azure Search will no longer be supported starting in api-version 2019-05-06-Preview. This was an "accidental feature" that we never intended to support. The reason you don't get any results is because the implicit conversion from Edm.Date to Edm.DateTimeOffset assumes a time of midnight UTC, whereas the dates in your index are 6 PM UTC.
We recommend explicitly providing the time and offset (or Z for UTC) in filters to avoid this problem.
If you want Azure Search to natively support fields and filters of type Edm.Date, please vote for this User Voice suggestion.
Date and time values represented in the OData V4 format: yyyy-MM-ddTHH:mm:ss.fffZ or yyyy-MM-ddTHH:mm:ss.fff[+|-]HH:mm. Precision of DateTimeOffset fields is limited to milliseconds. If you upload DateTimeOffset values with sub-millisecond precision, the value returned will be rounded up to milliseconds.
When you upload DateTimeOffset values with time zone information to your index, Azure Search normalizes these values to UTC.
Refer to supported data types in Azure search.
I am new to solr and this is my first attempt at indexing solr data, I am getting the following exception while indexing,
org.apache.solr.common.SolrException: Invalid Date String:'2011-01-07'
at org.apache.solr.schema.DateField.parseMath(DateField.java:165)
at org.apache.solr.schema.TrieDateField.createField(TrieDateField.java:169)
at org.apache.solr.schema.SchemaField.createField(SchemaField.java:98)
at org.apache.solr.update.DocumentBuilder.addField(DocumentBuilder.java:204)
at org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:277)
I understand from reading some articles that Solr stores time only in UTC, this is the query i am trying to index,
Select id,text,'language',links,tweetType,source,location, bio,url,utcOffset,timeZone,frenCnt,createdAt,createdOnGMT,createdOnServerTime,follCnt,favCnt,totStatusCnt,usrCrtDate,humanSentiment,replied,replyMsg,classified,locationDetail, geonameid,country,continent,placeLongitude,placeLatitude,listedCnt,hashtag,mentions,senderInfScr, createdOnGMTDate,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+05:30'),'%Y-%m-%d') as IST,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+01:00'),'%Y-%m-%d') as ECT,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+02:00'),'%Y-%m-%d') as EET,DATE_FORMAT(CONVERT_TZ(createdOnGMTDate,'+00:00','+03:30'),'%Y-%m-%d') as MET,sign(classified) as sentiment from
Why i am doing this timezone conversion is because i need to group results by the user timezone. How can i achieve this?
Regards,
Rohit
Solr dates must be in the form 1995-12-31T23:59:59Z. You're only giving the date part, but not the time.
See the DateField javadocs for more details.
Date faceting is entirely driven by query params, so if we index your events using the "true" time that they happend at (formatted as a string in UTC) you can then select your date ranges using whatever timezone offset is specified by your user at query time as a UTC offset.
facet.range = dateField
facet.range.start = 2011-01-01T00:00:00Z+${useroffset}MINUTES
facet.range.gap = +1DAY
This would return result in the users timezone and there is actually no need to timezone conversion the query and indexing that column separately.
Regards,
Rohit
Credit For Answer: Chris Hostetter (Solr User Group )