Using Solr 8.0.0, with each document holding a start timestamp field and an end timestamp field, how would I query in a way that returns just the duration between these dates? So I would be going for an equation like this:
(Endtime - Starttime) - 500 seconds = 23 seconds over expected duration.
But getting the result across all documents in the collection.
Would this be the subject of a streaming expression? Any example code you can give? I specifically want to keep this calculation load within the SolrCloud.
You can use a function query. The ms function gives you the difference in milliseconds between two dates. You can use sub to subtract 500 seconds from that number.
You can use the frange query parser to filter documents that match a given range. Which means we end up with something like:
q={!frange l=0}sub(ms(endtime,starttime),500000)
Related
We have an API that queries an Influx database and a report functionality was implemented so the user can query data using a start and end date.
The problem is that when a longer period is chosen(usually more than 8 weeks), we get a timeout from influx, query takes around 13 seconds to run. When the query returns a dataset successfully, we store that in cache.
The most time-consuming part of the query is probably comparison and averages we do, something like this:
SELECT mean("value") AS "mean", min("value") AS "min", max("value") AS "max"
FROM $MEASUREMENT
WHERE time >= $startDate AND time < $endDate
AND ("field" = 'myFieldValue' )
GROUP BY "tagname"
What would be the best approach to fix this? I can of course limit the amount of weeks the user can choose, but I guess that's not the ideal fix.
How would you approach this? Increase timeout? Batch query? Any database optimization to be able to run this faster?
In such cases where you allow user to select in days, I would suggest to have another table that stores the result (min, max and avg) of each day as a document. This table can be populated using some job after end of the day.
You can also think changing the document per day to per week or per month, based on how you plot the values. You can also add more fields like in your case, tagname and other fields.
Reason why this is superior to using a cache: When you use a cache, you can store the result of the query, so you have to compute for every different combination in realtime. However, in this case, the cumulative results are already available with much smaller dataset to compute.
Based on your query, I assume you are using InfluxDB v1.X. You could try Continuous Queries which are InfluxQL queries that run automatically and periodically on realtime data and store query results in a specified measurement.
In your case, for each report, you could generate a CQ and let your users to query it.
e.g.:
Step 1: create a CQ
CREATE CONTINUOUS QUERY "cq_basic_rp" ON "db"
BEGIN
SELECT mean("value") AS "mean", min("value") AS "min", max("value") AS "max"
INTO "mean_min_max"
FROM $MEASUREMENT
WHERE "field" = 'myFieldValue' // note that the time filter is not here
GROUP BY time(1h), "tagname" // here you can define the job interval
END
Step 2: Query against that CQ
SELECT * FROM "mean_min_max"
WHERE time >= $startDate AND time < $endDate // here you can pass the user's time filter
Since you already ask InfluxDB to run these aggregates continuously based on the specified interval, you should be able to trade space for time.
I am creating a report in SSRS that shows the duration of phone calls.
In my T-SQL script I am using:
CONVERT(VARCHAR,DATEADD(second,Call,0),108)AS[Call Duration]
which works nicely and shows the time as 00:03:20, for example.
However when I create a table in SSRS and try to sum all the different time values it just says #error in the report. I need the report to be able to add these time values up so I can give a total per switchboard operator. So for example if officer x took three calls and they all lasted 3 minutes then I'd need the total to say 00:09:00
Do you know of a way where I can display the total time spend rather than having to list each value separately? I can sum up the number of seconds for each call - so for example get a total of 540 seconds - but need to show this as hh:mm:ss
Thanks
The report is throwing an error because you are trying to sum up a varchar value. Rather than trying to format your data in your SQL query, simply return the values in their raw form to your SSRS report and let your presentation layer format the data for you.
Rather than using a dateadd, it seems your call length is already held within your Call column? If that is the case, simply return that column to your report, either as detail rows to be summed if the detail is required elsewhere in your report, or pre-aggregated in your SQL as this will perform better.
You can then format your Call Duration as follows:
=format(today().AddSeconds(Fields!Call.Value),"HH:mm:ss")
If you aren't aggregating your call seconds in your SQL query, you will need to do this in your expression:
=format(today().AddSeconds(sum(Fields!Call.Value)),"HH:mm:ss")
Obviously this method assumes you won't have any calls longer than 24 hours. If that is a possibility, you will need to calculate the hours, minutes and seconds to be concatenated together.
I'm trying to build a solr query for timecode values with the following format:
run_time:
00:25:00
00:30:00
01:00:00
I'd like to filter a range of timecodes. Everything 30 minutes or under, for example.
I've tried a couple of queries with no success.
run_time:[* TO 00:30:00]
run_time:[* TO 00\:30\:00]
run_time:[* TO *00:31:00Z]
Any thoughts?
Read here for more details about solr date operations:
run_time:[ NOW-30MINUTES TO NOW ]
Your use case appears to be searching for processes which ran for 30 minutes or less. Date field will not be the right fit here.
A better approach would be to store the run time information in a number field.
run_time:
25
30
60
The you can use a range query on this field to get desired results.
runtime:[0 TO 30]
This will fetch records having runtime between 0 to 30minutes including 0,30
What am I doing wrong in this query?
SELECT * FROM TreatmentPlanDetails
WHERE
accountId = 'ag5zfmRvbW9kZW50d2ViMnIRCxIIQWNjb3VudHMYtcjdAQw' AND
status = 'done' AND
category = 'chirurgia orale' AND
setDoneCalendarEventStartTimestamp >= [timestamp for 6 june 2012] AND
setDoneCalendarEventStartTimestamp <= [timestamp for 11 june 2012] AND
deleteStatus = 'notDeleted'
ORDER BY setDoneCalendarEventStartTimestamp ASC
I am not getting any record and I am sure there are records meeting the where clause conditions. To get the correct records I have to widen the timestamp interval by 1 millisecond. Is it normal? Furthermore, if I modify this query by removing the category filter, I am getting the correct results. This is definitely weird.
I also asked on google groups, but I got no answer. Anyway, for details:
https://groups.google.com/forum/?fromgroups#!searchin/google-appengine/query/google-appengine/ixPIvmhCS3g/d4OP91yTkrEJ
Let's talk specifically about creating timestamps to go into the query. What code are you using to create the timestamp record? Apparently that's important, because fuzzing with it a little bit affects the query. It may be relevant that in the datastore, timestamps are recorded as integers representing posix timestamps with microseconds, i.e. the number of microseconds since 1/1/1970 UTC (not counting leap seconds). It's also relevant that dates (i.e. without a time) are represented as midnight, i.e. the earliest time on that day. But please show us the exact code. (It may also be important to show the actual content of the record that you're attempting to retrieve.)
An aside that is not specific to your question: Entity property names count as part of your storage quota. If this is going to be a huge dataset, you might pay more $$ than you'd like for property names like setDoneCalendarEventStartTimestamp.
Because you write :
if I modify this query by removing the category filter, I am getting
the correct results
this probably means that the category was not indexed at the time you write the matching records to the data store. You have to re-write your records to the data store if you want them added to the newly created index.
I'm using solr 3.6 and I'm kinda stuck trying to perform a special query.
I'm actually using facets by date range, the face.date.gap is set to +1DAY. Of course, the facet is supposed to return the count of docs at a date range but I also need to get the sum of a special field at the same ranges used in facet. It's like I need to count how many votes I have daily monthly, weekly, whatever... it depends on the gap params.
Any ideas? Should I use the group.query or facet.query?
One suggestion I have is to treat the weeks, days separately, and index them. For ex. Today is part of 24th week. Another suggestion is not to rule out multiple searches to service one request. One to calculate all oth facets and one to return counts for given date range (based on search results from first query).