Filter datetime columns in SQL Server - sql-server

Question

I have a datetime column that has a 5 min interval between the next data, however I want to see if that column contains any time interval less than 5 mins, particularly 5 secs.
So for example:
one date would read 2018-05-04 19:21:46.000
the next row would read 2018-05-04 19:26:46.000
and 2018-05-04 19:31:46.000.
However, we sometimes get rows that read:
2018-05-04 19:36:46.000
then 2018-05-04 19:36:51.000
then 2018-05-04 19:36:56.000
What SQL script would be best to filter the column to distinguish the erroneous data (the 5 secs interval) from the correct data (5 min interval) especially in a table with thousands of rows?
Hi #Andrea, thanks for that. I have a couple of questions. What does the 'q' stand for? and when i rewrite the query as
SELECT ProductID, MyTimestamp, DATEDIFF(second, xMyTimestamp, MyTimestamp) as DIFFERENCE_IN_SECONDS
FROM (
SELECT *,
Lag(MyTimestamp) OVER (ORDER BY MyTimestamp, ProductID) as xMyTimestamp
FROM TableName
) q
WHERE xMyTimestamp IS NOT NULL and ProductID= 31928
I get this result which doesn't compute the time accurately.
+-----------+-------------------------+-----------------------+
| ProductID | MyTimestamp | DIFFERENCE_IN_SECONDS |
+-----------+-------------------------+-----------------------+
| 31928 | 2017-03-21 13:36:30.000 | 0 |
| 31928 | 2017-03-21 13:46:30.000 | 0 |
| 31928 | 2017-03-21 13:56:32.000 | 0 |
| 31928 | 2017-03-21 14:01:32.000 | 0 |
| 31928 | 2017-03-21 14:11:32.000 | 0 |
| 31928 | 2017-03-21 14:16:32.000 | 0 |
| 31928 | 2017-03-21 14:26:32.000 | 0 |
| 31928 | 2017-03-21 14:36:32.000 | 0 |
+-----------+-------------------------+-----------------------+
Any reason why

Answer 1

Since you are on 2014, you can use LEAD to compare the value of one row, to the value of the next.
declare #table table(id int identity(1,1), interval datetime)
insert into #table
values
('2018-05-04 19:21:46.000'),
('2018-05-04 19:26:46.000'),
('2018-05-04 19:31:46.000'),
('2018-05-04 19:36:46.000'),
('2018-05-04 19:36:51.000'),
('2018-05-04 19:36:56.000')
select
id
,interval
,issue_with_row = case
when
isnull(datediff(minute,interval,lead(interval) over (order by id, interval)),0) < 5
then 1
else 0
end
from #table
order by id
Or if you wanted to only see those,
;with cte as(
select
id
,interval
,issue_with_row = case
when
isnull(datediff(minute,interval,lead(interval) over (order by id, interval)),0) < 5
then 1
else 0
end
from #table)
select *
from cte
where issue_with_row = 1

Answer 2

You can use LAG:
declare #tmp table(MyTimestamp datetime)
insert into #tmp values
('2018-05-04 19:21:46.000')
,('2018-05-04 19:26:46.000')
,('2018-05-04 19:31:46.000')
,('2018-05-04 19:36:46.000')
,('2018-05-04 19:36:51.000')
,('2018-05-04 19:36:56.000')
SELECT DATEDIFF(second, xMyTimestamp, MyTimestamp) as DIFFERENCE_IN_SECONDS
FROM (
SELECT *,
LAG(MyTimestamp) OVER (ORDER BY MyTimestamp) xMyTimestamp
FROM #tmp
) q
WHERE xMyTimestamp IS NOT NULL
results:
So you should use it like this:
SELECT DATEDIFF(second, xMyTimestamp, MyTimestamp) as DIFFERENCE_IN_SECONDS
FROM (
SELECT *,
LAG(MyTimestamp) OVER (ORDER BY MyTimestamp) xMyTimestamp
FROM [YOUR_TABLE_NAME_HERE]
) q
WHERE xMyTimestamp IS NOT NULL
Edit
Here is another sample based on new data posted by OP:
declare #tmp table(ProductID int, MyTimestamp datetime)
insert into #tmp values
(31928, '2017-03-21 13:36:30.000')
,(31928, '2017-03-21 13:46:30.000')
,(31928, '2017-03-21 13:56:32.000')
,(31928, '2017-03-21 14:01:32.000')
,(31928, '2017-03-21 14:11:32.000')
,(31928, '2017-03-21 14:16:32.000')
,(31928, '2017-03-21 14:26:32.000')
,(31928, '2017-03-21 14:36:32.000')
SELECT ProductID
,MyTimestamp
,DATEDIFF(second, xMyTimestamp, MyTimestamp) AS DIFFERENCE_IN_SECONDS
FROM (
SELECT *
,Lag(MyTimestamp) OVER (
ORDER BY MyTimestamp
,ProductID
) AS xMyTimestamp
FROM #tmp
) q
WHERE xMyTimestamp IS NOT NULL
AND ProductID = 31928
Output:
Here you can check that the results are calculated correctly.