I am trying to extract the difference between two SQL DateTime values in seconds, with decimal places for some performance monitoring.
I have a table, "Pagelog" which has a "created" and "end" datetime. In the past I have been able to do the following:
SELECT DATEDIFF(ms, pagelog_created, pagelog_end)/1000.00 as pl_duration FROM pagelog
However I have started getting the following error:
Msg 535, Level 16, State 0, Line 1
The datediff function resulted in an overflow. The number of dateparts separating two date/time instances is too large. Try to use datediff with a less precise datepart.
I have seen numerous responses to this error stating that I should use a less precise unit of measurement. But this hardly helps when I need to distinguish between 2.1 seconds and 2.9 seconds, because DATEDIFF(s,..,..) will return INT results and lose the accuracy I need.
I originally thought that this had been caused by a few values in my table having a huge range but running this:
SELECT DATEDIFF(s, pagelog_created, pagelog_end) FROM pagelog
ORDER BY DATEDIFF(s, pagelog_created, pagelog_end) DESC
Returns a max value of 30837, which is 8.5 hours or 30,837,000 milliseconds, well within the range of a SQL INT as far as I know?
Any help would be much appreciated, as far as I can tell I have two options:
Somehow fix the problem with the data, finding the culprit values
Find a different way of calculating the difference between the values
Thanks!
The StackOverflow magic seems to have worked, despite spending hours on this problem last week, I re-read my question and have now solved this. I thought I'd update with the answer to help anyone else who has this problem.
The problem here was not that there was a large range, there was a negative range. Which obviously results in a negative overflow. It would have been helpful if the SQL Server error was a little more descriptive but it's not technically wrong.
So in my case, this was returning values:
SELECT * FROM pagelog
WHERE pagelog_created > pagelog_end
Either remove the values, or omit them from the initial result set!
Thanks to Ivan G and Andriy M for your responses too
You can try to avoid overflow like this:
DECLARE #dt1 DATETIME = '2013-01-01 00:00:00.000'
DECLARE #dt2 DATETIME = '2013-06-01 23:59:59.997'
SELECT DATEDIFF(DAY, CAST(#dt1 AS DATE), CAST(#dt2 AS DATE)) * 24 * 60 * 60
SELECT DATEDIFF(ms, CAST(#dt1 AS TIME), CAST(#dt2 AS TIME))/1000.0
SELECT DATEDIFF(DAY, CAST(#dt1 AS DATE), CAST(#dt2 AS DATE)) * 24 * 60 * 60
+ DATEDIFF(ms, CAST(#dt1 AS TIME), CAST(#dt2 AS TIME))/1000.0
First it gets number of seconds in whole days from the DATE portion of the DATETIME and then it adds number of seconds from the TIME portion, after that, it just adds them.
There won't be error because DATEDIFF for minimum and maximum time in TIME data type cannot produce overflow.
You could of course do something like this:
SELECT
DATEDIFF(ms, DATEADD(s, x.sec, pagelog_created), pagelog_end) * 0.001
+ x.sec AS pl_duration
FROM pagelog
CROSS APPLY (
SELECT DATEDIFF(s, pagelog_created, pagelog_end)
) x (sec)
;
As you can see, first, the difference in seconds between pagelog_created and pagelog_end is taken, then the seconds are added back to pagelog_created and the difference in milliseconds between that value and pagelog_end is calculated and added to the seconds.
However, since, as per your investigation, the table doesn't seem to have rows that could cause the overflow, I'd also double check whether that particular fragment was the source of the error.
with cte as(
select rownum = row_number() over(partition by T.TR_ID order by T.[date]),
T.* from [dbo].[TR_Events] T
)
select cte.[date],nex.[date],convert(varchar(10),datediff(s, cte.[date], nex.[date])/3600)+':'+
convert(varchar(10),datediff(s, cte.[date], nex.[date])%3600/60)+':'+
convert(varchar(10),(datediff(s,cte.[date], nex.[date])%60))
from cte
left join cte prev on prev.rownum = cte.rownum - 1
left join cte nex on nex.rownum = cte.rownum + 1
Related
We are attempting to calculate a rolling average and have tried to convert numerous SO answers to solve the problem. To this point we are still unsuccessful.
What we've tried:
Here are some of the SO answers we have considered.
SQL Server: How to get a rolling sum over 3 days for different customers within same table
SQL Query for 7 Day Rolling Average in SQL Server
T-SQL calculate moving average
Our latest attempt has been to modify one of the solutions (#4) found here.
https://www.red-gate.com/simple-talk/sql/t-sql-programming/calculating-values-within-a-rolling-window-in-transact-sql/
Example:
Here is an example in SQL Fiddle: http://sqlfiddle.com/#!6/4570a/17
In the fiddle, we are still trying to get the SUM to work right but ultimately we are trying to get the average.
The end goal
Using the Fiddle example, we need to find the difference between Value1 and ComparisonValue1 and present it as Diff1. When a row has no Value1 available, we need to estimate it by taking the average of the last two Diff1 values and then add it to the ComparisonValue1 for that row.
With the correct query, the result would look like this:
GroupID Number ComparisonValue1 Diff1 Value1
5 10 54.78 2.41 57.19
5 11 55.91 2.62 58.53
5 12 55.93 2.78 58.71
5 13 56.54 2.7 59.24
5 14 56.14 2.74 58.88
5 15 55.57 2.72 58.29
5 16 55.26 2.73 57.99
Question: is it possible to calculate this average when it could potentially factor into the average of the following rows?
Update:
Added a VIEW to the Fiddle schema to simplify the final query.
Updated the query to include the new rolling average for Diff1 (column Diff1Last2Avg). This rolling average works great until we run into nulls in the Value1 column. This is where we need to insert the estimate.
Updated the query to include the estimate that should be used when there is no Value1 (column Value1Estimate). This is working great and would be perfect if we could use the estimate in place of NULL in the Value1 column. Since the Diff1 column reflects the difference between Value1 (or its estimate) and ComparisonValue1, including the Estimate would fill in all the NULL values in Diff1. This in turn would continue to allow the Estimates of future rows to be calculated. It gets confusing at this point, but still hacking away at it. Any ideas?
Credit for the idea goes to this answer: https://stackoverflow.com/a/35152131/6305294 from #JesúsLópez
I have included comments in the code to explain it.
UPDATE
I have corrected the query based on comments.
I have swapped numbers in minuend and subtrahend to get difference as a positive number.
Removed Diff2Ago column.
Results of the query now exactly match your sample output.
;WITH cte AS
(
-- This is similar to your ItemWithComparison view
SELECT i.Number, i.Value1, i2.Value1 AS ComparisonValue1,
-- Calculated Differences; NULL will be returned when i.Value1 is NULL
CONVERT( DECIMAL( 10, 3 ), i.Value1 - i2.Value1 ) AS Diff
FROM Item AS i
LEFT JOIN [Group] AS G ON g.ID = i.GroupID
LEFT JOIN Item AS i2 ON i2.GroupID = g.ComparisonGroupID AND i2.Number = i.Number
WHERE NOT i2.Id IS NULL
),
cte2 AS(
/*
Start with the first number
Note if you do not have at least 2 consecutive numbers (in cte) with non-NULL Diff value and therefore Diff1Ago or Diff2Ago are NULL then everything else will not work;
You may need to add additional logic to handle these cases */
SELECT TOP 1 -- start with the 1st number (see ORDER BY)
a.Number, a.Value1, a.ComparisonValue1, a.Diff, b.Diff AS Diff1Ago
FROM cte AS a
-- "1 number ago"
LEFT JOIN cte AS b ON a.Number - 1 = b.Number
WHERE NOT a.Value1 IS NULL
ORDER BY a.Number
UNION ALL
SELECT b.Number, b.Value1, b.ComparisonValue1,
( CASE
WHEN NOT b.Value1 IS NULL THEN b.Diff
ELSE CONVERT( DECIMAL( 10, 3 ), ( a.Diff + a.Diff1Ago ) / 2.0 )
END ) AS Diff,
a.Diff AS Diff1Ago
FROM cte2 AS a
INNER JOIN cte AS b ON a.Number + 1 = b.Number
)
SELECT *, ( CASE WHEN Value1 IS NULL THEN ComparisonValue1 + Diff ELSE Value1 END ) AS NewValue1
FROM cte2 OPTION( MAXRECURSION 0 );
Limitations:
this solution works well only when you need to consider small number of preceding values.
i have the following code:
select COUNT(STL.ITEID)
from STORETRADELINES STL
left join FINTRADE FT ON STL.FTRID=FT.ID
where RIGHT(CONVERT(datetime, FT.FTRDATE, 3), 5) < RIGHT(CONVERT(datetime, GETDATE(), 3), 5)
and FT.FTRDATE > CONVERT(datetime,'01.01.2017',103)
I want to select all the documents that fall between 01.01.2017 and 30.04.2017 (always, the last day of the previous month).
Seem's that this way is not good, because is returning all the docs from 01.01 until today.
Where am i wrong ? Btw, i use sql server 2008.
Thank you
The problem is, you're doing string comparisons. Use date comparisons instead:
select COUNT(STL.ITEID)
from STORETRADELINES STL
left join FINTRADE FT ON STL.FTRID=FT.ID
where FT.FTRDATE < DATEADD(month,DATEDIFF(month,0,GETDATE()),0)
and FT.FTRDATE >= '20170101'
Here, the DATEADD/DATEDIFF pair of calls are rounding the current date down to the start of the current month.
You can use a similar DATEADD/DATEDIFF pair for your start-of-period condition, using year instead of month if you want this query to be generic for any current year
The reason you're "stringly-typed" code doesn't work is that your CONVERT calls are converting datetime values to datetime. The style parameter is ignored here since datetimes don't have a format. You then get an uncontrolled implicit conversion of the datetimes into varchar so that RIGHT can work, and it's that conversion that you should have tried to control.
As it is, this:
select RIGHT(CONVERT(datetime, GETDATE(), 3), 5)
Generates:
:50AM
Which you can hopefully see is nothing like what you wanted.
dateadd(mm,datediff(mm, 0 , current_timestamp),0) will return the first day of current month.
Your query could be
SELECT COUNT(STL.ITEID)
FROM STORETRADELINES STL
left join FINTRADE FT ON STL.FTRID=FT.ID
where
FT.FTRDATE >= '2017-01-01' AND FT.FTRDATE < dateadd(mm,datediff(mm, 0 , current_timestamp),0)
What's an alternative to getting a distinct number of dates, say all the dates for September:
9/1/2016
9/2/2016
9/3/2016
and apply each value to a query. Say something like:
Select GuitarId,GuitarBrand
From GuitarSales
Where GuitarDate = #date
I don't want to use a cursor, is there an alternative to doing this?
I tried a CTE but even then I'd have to apply the cursor for each date.
If you want all the dates for a month you can use
Select GuitarId,GuitarBrand
From GuitarSales
Where month(GuitarDate) = 9
and year(GuitarDate) = 2016;
If I understand you correctly, you need a list of all dates in September. This is a quick solution to get a gapless list of all days in September: In your query you can use this as source and LEFT JOIN your actual data.
WITH RunningNumbers AS
(
SELECT TOP(30) ROW_NUMBER() OVER(ORDER BY (SELECT NULL))-1 AS Nr
FROM sys.objects
)
SELECT {d'2016-09-01'}+Nr AS RunningDate
FROM RunningNumbers
There are many examples, how you can create a tally table on the fly. Small numbers (like 30 in this example) can be taken easily from any table with sufficient rows.
If you need this more often you might think about a Numbers-Table
a related question: https://stackoverflow.com/a/39387790/5089204
create a persitant numbers table with a lot of usefull side data: https://stackoverflow.com/a/32474751/5089204
Assuming you have an index on GuitarDate here is a way you can create a SARGable where predicate so you can still leverage the speed of using an index seek.
declare #date datetime = '2016-09-10' --just to demonstrate starting with September 10, 2016
select gs.GuitarId
, gs.GuitarBrand
From GuitarSales gs
where gs.GuitarDate >= dateadd(month, datediff(month, 0, #date), 0) --beginning of the month for #date
and gs.GuitarDate < dateadd(month, datediff(month, 0, #date) + 1, 0) --beginning of next month
I have following statement
select pkid
from AttendancePosting
where datename(dw,AttDate) = 'Sunday' and empid=4 and attdate='2015-12-13'
group by PKId,timeout
--having 9=9
having cast(sum((datepart(minute, timeout)))/2 as float )+''=cast(datepart(minute,timeout) as float) +''
The problem is
having cast(sum((datepart(minute, timeout)))/2 as float )+''=cast(datepart(minute,timeout) as float) +''
Not working. both cast(sum((datepart(minute, timeout)))/2 as float ) and cast(datepart(minute,timeout) as float) bring the same value but still the select statement is not fetching any records, both returns 9
I have checked it like this
select pkid
from AttendancePosting
where datename(dw,AttDate) = 'Sunday' and empid=4 and attdate='2015-12-13'
group by PKId,timeout
having 9=9
And its bringing records, Any help will be appreciated.
First, your statement datepart(minute, timeout)/2 is going to return an integer. You can make SQL Server get more precise by being more precise like this datepart(minute, timeout)/2..
Second, floating point numbers are an approximation. You would do better to use ROUND() and specify the number of decimal places you think is appropriate. For example: round(sum((datepart(minute, timeout)))/2.0, 3).
In SQL Server, I want to show all items for a particular day. What is the best way to do this: DATEPART, BETWEEN, OR >= + < ? All 3 should work functionally, but are some of them better/worse for performance?
1)
SELECT *
FROM BlogPosts
WHERE DATEPART(yyyy,BlogPostDate) = 2011
AND DATEPART(m,BlogPostDate) = 5
AND DATEPART(d,BlogPostDate) = 7
2)
SELECT *
FROM BlogPosts
WHERE BlogPostDate BETWEEN '2011-05-07' AND '2011-05-07 23:59:59'
3)
SELECT *
FROM BlogPosts
WHERE BlogPostDate >= '2011-05-07'
AND BlogPostDate < '2011-05-08'
Out of the options you have presented. Number 3 as it is both sargable and will work.
Number 1 is not sargable (can't use an index).
Number 2 will miss valid datetimes such as 2011-05-07 23:59:59.997 (and the exact maximum datetime that you would have to use here would vary between smalldatetime, datetime, and datetime2(x))
If you are on SQL Server 2008 an alternative option which is sargable despite appearances is
SELECT *
FROM BlogPosts
WHERE CAST(BlogPostDate AS Date) = '2011-05-07'
However I would still opt for your Number 3 with the >= .... < as being more efficient both in terms of the range seeked and also potentially making better use of column statistics. More details of that here.
Personally I prefer the DateDiff technique below.
DECLARE #selectedDate datetime = '2011-05-07'
SELECT *
FROM BlogPosts
WHERE DateDiff(DAY,BlogPostDate, #selectedDate) = 0
Options 1) and 2) will both work.
Option 3 will not work as you expect, since datetime has a higher precision than seconds. So the where clause:
WHERE BlogPostDate BETWEEN '2011-05-07' AND '2011-05-07 23:59:59'
Would incorrectly filter out:
'2011-05-07 23:59:59.500'
Datetime values are rounded to increments of .000, .003, or .007 seconds. So this WHERE clause would work:
WHERE BlogPostDate BETWEEN '2011-05-07' AND '2011-05-07 23:59:59.997'