Getting duration is seconds between datetime rows in SSIS - sql-server

I have an event-based csv file that logs an event along with the time of execution. I need to be able to get the duration between the events by taking the DateTime row and subtracting the DateTime row below it to give me the total duration of that particular event in secs.
So take the sample table below:
Date/Time Event CF_ID EventDuration(s)
04/11/2022 08:00:09 Login 521
04/11/2022 08:01:29 Inxt 426
04/11/2022 08:23:57 Rgal 731
04/11/2022 08:24:08 hold 78
After transformation, it should look something like this:
Date/Time Event CF_ID EventDuration(s)
04/11/2022 08:00:09 Login 521 80
04/11/2022 08:01:29 Call 426 1348
04/11/2022 08:23:57 DAB 731 11
04/11/2022 08:24:08 hold 78
I can probably achieve this in SQL with relative ease, however, I need to be able to use an SSIS transformation to do this such as a derived column. Can this be achieved in SSIS?

CREATE TABLE MyTable(DateTime datetime, Events varchar(255), CF_ID INT)
GO
✓
INSERT INTO MyTable (DateTime,Events,CF_ID) VALUES
('04/11/2022 08:00:09' , 'Login', 521 ),
('04/11/2022 08:01:29' , 'Inxt', 426 ),
('04/11/2022 08:23:57' , 'Rgal', 731 ),
('04/11/2022 08:24:08' , 'hold', 78 )
GO
4 rows affected
WITH CTE AS
(SELECT
DateTime,
Events,
CF_ID,
(DATEPART(SECOND, DateTime) +
60 * DATEPART(MINUTE, DateTime) +
3600 * DATEPART(HOUR, DateTime))
AS ConvertToSeconds
FROM MyTable)
SELECT
DateTime,
Events,
CF_ID,
LEAD(ConvertToSeconds) OVER( ORDER BY DateTime) - ConvertToSeconds
FROM CTE
ORDER BY DateTime
GO
DateTime | Events | CF_ID | (No column name)
:---------------------- | :----- | ----: | ---------------:
2022-04-11 08:00:09.000 | Login | 521 | 80
2022-04-11 08:01:29.000 | Inxt | 426 | 1348
2022-04-11 08:23:57.000 | Rgal | 731 | 11
2022-04-11 08:24:08.000 | hold | 78 | null
db<>fiddle here

Related

SQL Server, Merging 2 Rows into 1 and limit row grouping

I am currently making a SQL Query to access data in a table called "Alarms". This table is set up as in the Following Format:
AlarmNumber | Time | AlarmState
-------------|-------|-----------
1046 | 10:30 | 0
1045 | 10:25 | 1
1044 | 10:24 | 0
1046 | 10:24 | 1
1046 | 10:23 | 0
1046 | 10:22 | 1
What I would like to achieve is to sort the alarms Into the Following Format
The Goal is to display the Alarm Start Time, Alarm Stop Time and Alarm Active Time (Alarm End Time - Alarm Start Time)
AlarmNumber | AlarmStartTime | AlarmEndTime | AlarmActiveTime
-------------|-----------------|--------------|----------------
1046 | 10:24 | 10:30 | 00:02
1045 | 10:24 | - | 10:24 + Current Time
1044 | Shift Start Time| 10:30 |10:30 - Shift Start Time
1046 | 10:22 | 10:23 | 00:01
My current code is the following (Note: _Global_Vars is a table with Timezones):
SELECT
TODATETIMEOFFSET([ALARM_START_TIME],0) AT TIME ZONE (SELECT g.LocalTimeZone FROM _Global_Vars as g) AS [ALARM_START_TIME],
TODATETIMEOFFSET(ALARM_FINISH_TIME,0) AT TIME ZONE (SELECT g.LocalTimeZone FROM _Global_Vars as g) AS [ALARM_FINISH_TIME],
DATEDIFF(SS, [ALARM_START_TIME], [ALARM_FINISH_TIME]),
sub.AlarmNumber
FROM
(
SELECT
(a.[Time]) AS AlarmTime,
(a.[AlarmNumber]+1) as AlarmNumber,
(CASE WHEN a.[AlarmState] = 1 THEN a.[Time] END) [ALARM_START_TIME],
(CASE WHEN a.[AlarmState] = 0 THEN a.[Time] END) [ALARM_FINISH_TIME]
FROM [Alarms] as a
WHERE (a.[Time] > DATEADD(mi, - 60.0 * 12, GETUTCDATE()))
)`
The issue at the moment is that if I use MAX in front of the CASE and GROUP BY AlarmNumber, it combines all of the values for AlarmNumber into a single row where I would like it to have multiple instances of Alarmnumber if the Alarm occurs multiple times
I am a novice regarding writing SQL Queries so any help would be great.
I will post just part of the solution (containing just first three colmns) since you not clearly specified the goal content for the last column.
SELECT t.alarmNumber,
isnull(MAX(CASE WHEN t.alarmstate = 1 THEN CAST(t.[time] as varchar(20)) END), 'Shift Start Time') AlarmStartTime,
isnull(MAX(CASE WHEN t.alarmstate = 0 THEN CAST(t.[time] as varchar(20)) END), 'Current Time') AlarmEndTime
FROM
(
SELECT *, row_number() over (partition by alarmnumber, alarmstate order by [time]) al_group
FROM Alarms
) t
GROUP BY t.alarmNumber, t.al_group
demo

Grouping rows to minimise deviation

I have a Employee Wages table like this, with their EmpID and their wages.
EmpId | Wages
================
101 | 1280
102 | 1600
103 | 1400
104 | 1401
105 | 1430
106 | 1300
I need to write a Stored Procedure in SQL Server, to group the Employees according to their wages, such that similar salaried people are in groups together and the deviations within the group is as minimum as possible.
There are no other conditions or rules mentioned.
The output should look like this
EmpId | Wages | Group
=======================
101 | 1280 | 1
106 | 1300 | 1
103 | 1400 | 2
104 | 1401 | 2
105 | 1430 | 2
102 | 1600 | 3
You can use a query like the following:
SELECT EmpId, Wages,
DENSE_RANK() OVER (ORDER BY CAST(Wages - t.min_wage AS INT) / 100) AS grp
FROM mytable
CROSS JOIN (SELECT MIN(Wages) AS min_wage FROM mytable) AS t
The query calculates the distance of each wage from the minimum wage and then uses integer division by 100 in order to place records in slices. So all records that have a deviation that is between 0 - 99 off the minimum wage are placed in the first slice. The second slice contains records off by 100 - 199 from the minimum wage, etc.
You can for +-30 deviation as the below:
DECLARE #Tbl TABLE (EmpId INT, Wages INT)
INSERT INTO #Tbl
VALUES
(99, 99),
(100, 101),
(101, 1280),
(102, 1600),
(103, 1400),
(104, 1401),
(105, 1430),
(106, 1300)
;WITH CTE AS ( SELECT *, ROW_NUMBER() OVER (ORDER BY Wages) AS RowId FROM #Tbl )
SELECT
A.EmpId ,
A.Wages ,
DENSE_RANK() OVER (ORDER BY MIN(B.RowId)) [Group]
FROM
CTE A CROSS JOIN CTE B
WHERE
ABS(B.Wages - A.Wages) BETWEEN 0 AND 30 -- Here +-30
GROUP BY A.EmpId, A.Wages
ORDER BY A.Wages
Result:
EmpId Wages Group
----------- ----------- --------------------
99 99 1
100 101 1
101 1280 2
106 1300 2
103 1400 3
104 1401 3
105 1430 3
102 1600 4

select query to get first row from rows having multiple id's.(without partition by)

id date amount documentNo paperID
1 2015/10/15 500 1234 34
1 2015/10/15 100 1332 33
2 2015/10/13 200 1302 21
2 2015/10/13 400 1332 33
3 2015/11/23 500 1332 43
I should get the output as:
id date amount documentNo paperID
1 2015/10/15 500 1234 34
2 2015/10/13 200 1302 21
3 2015/11/23 500 1332 43
Please suggest a simple select query to fetch only one row without partition by. Note: the date remain same for a particular id.
Try a null-self-join. Basically you are comparing each row to some other version of that row ,but, via an inequality (here I have used documentNo) you end-up with a single row that has no match.
See this SQL Fiddle
MySQL 5.6 Schema Setup:
CREATE TABLE Table1
(`id` int, `date` datetime, `amount` int, `documentNo` int, `paperID` int)
;
INSERT INTO Table1
(`id`, `date`, `amount`, `documentNo`, `paperID`)
VALUES
(1, '2015-10-15 00:00:00', 500, 1234, 34),
(1, '2015-10-15 00:00:00', 100, 1332, 33),
(2, '2015-10-13 00:00:00', 200, 1302, 21),
(2, '2015-10-13 00:00:00', 400, 1332, 33),
(3, '2015-11-23 00:00:00', 500, 1332, 43)
;
Query 1:
SELECT
t1.*
FROM table1 AS t1
LEFT OUTER JOIN table1 AS t2 ON t1.id = t2.id
AND t1.date = t2.date
AND t2.documentNo < t1.documentNo
WHERE t2.ID IS NULL
Results:
| id | date | amount | documentNo | paperID |
|----|----------------------------|--------|------------|---------|
| 1 | October, 15 2015 00:00:00 | 500 | 1234 | 34 |
| 2 | October, 13 2015 00:00:00 | 200 | 1302 | 21 |
| 3 | November, 23 2015 00:00:00 | 500 | 1332 | 43 |
EDIT: There are several approaches to this problem even without windowing functions such as row_number() , here is a previous answer covering some MySQL specific alternatives.

How to create a date range from a list of dates

I've tried searching for an answer to my question but all the scenarios that have been covered previously are slightly different. Apologies If I've missed one that does answer this.
I have a table that looks like this:
ID Date
35 2015-06-01
35 2015-06-02
35 2015-06-03
35 2015-06-06
40 2015-06-07
40 2015-06-08
40 2015-06-09
40 2015-06-10
40 2015-06-13
35 2015-06-14
35 2015-06-15
35 2015-06-16
35 2015-06-17
And I would like to get it in the form:
ID DateFrom DateTo
35 2015-06-01 2015-06-07
40 2015-06-07 2015-06-14
35 2015-06-14 2015-06-18
However, every solution that I can think of or have seen (row_number(), min / max, lag etc) only returns the following
ID DateFrom DateTo
35 2015-06-01 2015-06-18
40 2015-06-07 2015-06-14
i.e the date range for ID 35 is taking the min and max of the dates from the first table. It seems like this should be easy but I'm completely stuck.
Thanks,
Tom
You can do this using ROW_NUMBER with MIN and MAX:
SQL Fiddle
WITH CteGrp AS(
SELECT *,
grp = ROW_NUMBER() OVER(ORDER BY Date)
- ROW_NUMBER() OVER(PARTITION BY ID ORDER BY Date)
FROM tbl
)
SELECT
ID,
DateFrom = MIN([Date]),
DateTo = MAX([Date])
FROM CteGrp
GROUP BY ID, grp
ORDER BY DateFrom
RESULT
| ID | DateFrom | DateTo |
|----|------------|------------|
| 35 | 2015-06-01 | 2015-06-06 |
| 40 | 2015-06-07 | 2015-06-13 |
| 35 | 2015-06-14 | 2015-06-17 |

sql query to delete only one duplicate row

I've a table with some duplicate rows in it. I want to delete only one duplicate row.
For example I'v 9 duplicate rows so should delete only one row and should show 8 remaining rows.
example
date calling called duration timestampp
2012-06-19 10:22:45.000 165 218 155 1.9 121
2012-06-19 10:22:45.000 165 218 155 1.9 121
2012-06-19 10:22:45.000 165 218 155 1.9 121
2012-06-19 10:22:45.000 165 218 155 1.9 121
from above date should delete only one row and should show 3 rows
2012-06-19 10:22:45.000 165 218 155 1.9 100
2012-06-19 10:22:45.000 165 218 155 1.9 100
2012-06-19 10:22:45.000 165 218 155 1.9 100
from above date should delete only one row and should show 2 rows
How can I do this?
This solution allows you to delete one row from each set of duplicates (rather than just handling a single block of duplicates at a time):
;WITH x AS
(
SELECT [date], rn = ROW_NUMBER() OVER (PARTITION BY
[date], calling, called, duration, [timestamp]
ORDER BY [date])
FROM dbo.UnspecifiedTableName
)
DELETE x WHERE rn = 2;
As an aside, both [date] and [timestamp] are terrible choices for column names...
For SQL Server 2005+ you can do the following:
;WITH CTE AS
(
SELECT *,
ROW_NUMBER() OVER(PARTITION BY [date], calling, called, duration, [timestamp] ORDER BY 1) RN
FROM YourTable
)
DELETE FROM CTE
WHERE RN = 2
Do you have a primary key on the table?
What makes a row a duplicate? Same time? same date? all columns being the same?
If you have a primary key you can use the TOP function to select only one record and delete that one row:
Delete from [tablename] where id in (select top 1 id from [tablename] where [clause])
If you don't mind the order of these rows there is a command in MySQL:
DELETE TOP (numberOfRowsToDelete) FROM db.tablename WHERE {condition for ex id = 5};
Since I don't have the schema, I'd a possible solution in steps:
Apply a row number to the select of all columns
Make a group by with those columns and delete the min(rownumber) in each group
Edit:
The rownumber is in a inner query and will have the rownumber incrementing in all rows. In the outer query I make the group by of the inner query and select the min(rownumber) for each group. Since each group is composed by duplicated rows, I then remove the min(rownumber) for each group.
using LIMIT 1 will help you delete only 1 ROW that matches your DELETE query:
DELETE FROM `table_name` WHERE `column_name`='value' LIMIT 1;
BEFORE:
+----------------------+
| id | column_name |
+-----+----------------+
| 1 | value |
+-----+----------------+
| 2 | value |
+-----+----------------+
| 3 | value |
+-----+----------------+
| 4 | value |
+-----+----------------+
AFTER:
+----------------------+
| id | column_name |
+-----+----------------+
| 1 | value |
+-----+----------------+
| 2 | value |
+-----+----------------+
| 3 | value |
+-----+----------------+

Resources