Group by datetime minutes - Show minutes with 0 entries in database? - sql-server

I'm trying to write a T-SQL query that looks for amount of entries in a table, over a certain period of time, and then group them by minute (entries/minute).
But I cant get it to show the minutes with 0 entries.
My query looks like:
select
[Minute],
Files
from
(
select
(DATEDIFF(MINUTE, '2017-02-02 17:00:00.000', RegTime) / 1) as [Minute] ,
count(*) as Files
from TransferLog
WHERE RegTime BETWEEN '2017-02-02 17:00:00.000' AND '2017-02-03 04:00:00.000'
group BY
(DATEDIFF(MINUTE, '2017-02-02 17:00:00.000', RegTime) / 1)
) x order by [Minute]
Any Ideas?
Edit1. Excepted output would show minutes with no entries in the table. Also, the query is rather unusable when this occurs since I can't know which minute had no data. For example if I want data from 800 minutes but 20 of the minutes had no data written to the table this would only show 780 minutes worth of data. Is there anyway to sort the data or modify the query for this purpose?
Expected output:
Minute | Files
0 685
1 0
2 672
3 0
4 415
5 434
6 746
-
Current Output:
Minute | Files
0 685
1 672
2 415
3 434
4 746

You can create a temp table holding all the minutes you need from 1 to the maximum minute you have (for example using this tecnique), then you can left join your data table with the temp table replacing null values with zeros.
Thus you should have all the minutes listed with your values where you have data and zeros elsewhere.
Here is the code:
declare #max_minute int
declare #dataTable table ([Minute] int, Files int)
--insert your data into a temp table
insert into #dataTable
select [Minute], Files
from
(select
(DATEDIFF(MINUTE, '2017-02-02 17:00:00.000', RegTime) / 1) as [Minute] ,
count(*) as Files
from TransferLog
WHERE RegTime BETWEEN '2017-02-02 17:00:00.000' AND '2017-02-03 04:00:00.000'
group BY
(DATEDIFF(MINUTE, '2017-02-02 17:00:00.000', RegTime) / 1)
) x order by [Minute]
--calculate maximum minute from your data
select #max_minute = max([Minute]) from #dataTable
;
WITH gen AS (
SELECT 1 AS Minute
UNION ALL
SELECT Minute+1 FROM gen WHERE Minute+1<=#max_minute
)
SELECT M.Minute, isnull(D.Files,0) as Files
FROM gen M left join #dataTable D on M.Minute = D.Minute
option (maxrecursion 10000)

Related

Accumulative Update for previous records

I have table that shows these information
Month NewClients OnHoldClients
5-2017 10 2
6-2017 16 4
7-2017 11 1
8-2017 15 6
9-2017 18 7
I am trying to find the accumulative total for each month
which is
(NewClients - OnHoldClients) + Previous Month Total
Something like this
Month NewClients OnHoldClients Total
5-2017 10 2 8
6-2017 16 4 20
7-2017 11 1 30
8-2017 15 6 39
9-2017 18 7 50
the query i tried to build was something like this but I think should be an easier way to do that
UPDATE MyTable
SET Total = (SELECT TOP 1 Total FROM MyTable B WHERE B.Month < A.Month) + NewClients - OnHoldClients
FROM MyTable A
Before we begin, note the mere fact that you're facing such calculative problem is a symptom that maybe you don't have the best possible design. Normally for this purpose calculated values are being stored along the way as the records are inserted. So i'd say you'd better have a total field to begin with and calculate it as records amass.
Now let's get down to the problem at hand. i composed a query which does that nicely but it's a bit verbose due to recursive nature of the problem. However, it yields the exact expected result:
DECLARE #dmin AS date = (SELECT min(mt.[Month]) from dbo.MyTable mt);
;WITH cte(_Month, _Total) AS (
SELECT mt.[Month] AS _Month, (mt.NewClients - mt.OnHoldClients) AS _Total
FROM dbo.MyTable mt
WHERE mt.[Month] = #dmin
UNION ALL
SELECT mt.[Month] AS _Month, ((mt.NewClients - mt.OnHoldClients) + ccc._Total) AS _Total
FROM dbo.MyTable mt
CROSS APPLY (SELECT cc._Total FROM (SELECT c._Total,
CAST((row_number() OVER (ORDER BY c._Month DESC)) AS int) as _Rank
FROM cte c WHERE c._Month < mt.[Month]) as cc
WHERE cc._Rank = 1) AS ccc
WHERE mt.[Month] > #dmin
)
SELECT c._Month, max(c._Total) AS Total
FROM cte c
GROUP BY c._Month
It is a recursive CTE structure that goes about each record all along the way to the initial month and adds up to the final Total value. This query only includes Month and Total fields but you can easily add the other 2 to the list of projection.
Try this
;WITH CTE([Month],NewClients,OnHoldClients)
AS
(
SELECT '5-2017',10,2 UNION ALL
SELECT '6-2017',16,4 UNION ALL
SELECT '7-2017',11,1 UNION ALL
SELECT '8-2017',15,6 UNION ALL
SELECT '9-2017',18,7
)
SELECT [Month],
NewClients,
OnHoldClients,
SUM(MonthTotal)OVER( ORDER BY [Month]) AS Total
FROM
(
SELECT [Month],
NewClients,
OnHoldClients,
SUM(NewClients-OnHoldClients)OVER(PArtition by [Month] Order by [Month]) AS MonthTotal
FROM CTE
)dt
Result,Demo:http://rextester.com/DKLG54359
Month NewClients OnHoldClients Total
--------------------------------------------
5-2017 10 2 8
6-2017 16 4 20
7-2017 11 1 30
8-2017 15 6 39
9-2017 18 7 50

select data, grouped as Histogram

I have data in this format:
CREATE TABLE data(y int)
INSERT INTO data VALUES ((1))
INSERT INTO data VALUES ((55555))
INSERT INTO data VALUES ((55555))
INSERT INTO data VALUES ((99999))
I want to create a histogram, for to get a rough overview, of how my data is distributed. I am thinking of this format as output:
lowerBoundary upperBoundary y
------------- ------------- -----------
0 9999 1
10000 19999 0
20000 29999 0
30000 39999 0
40000 49999 0
50000 59999 2
60000 69999 0
70000 79999 0
80000 89999 0
90000 99999 1
You will have to create a table of numbers, so that the 0-rows will be displayed correctly. Then you can calculate lower and upper boundary of each "group".
Example SQL:
SELECT lowerBoundary, upperBoundary, COUNT(d.y) AS y
FROM (
SELECT n*10000 AS lowerBoundary, (n+1)*10000-1 AS upperBoundary
FROM (
-- Selects possible groups. Make this big enough for your data.
SELECT ones.n + 10*tens.n + 100*hundreds.n AS n
FROM (VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) ones(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) tens(n),
(VALUES(0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) hundreds(n)
) numbersTable
) boundaries
-- join with data
LEFT JOIN data d
ON d.y BETWEEN lowerBoundary AND upperBoundary
-- avoid trailing '0' rows
WHERE lowerBoundary <= (SELECT MAX(d.y) FROM data d)
GROUP BY lowerBoundary, upperBoundary
ORDER BY 1
Click here to run this skript at SQL-Fiddle
Another option...
I use a TVF to generate dynamic ranges. Being a single-statement function, it is extremely fast. Furthermore, if you can't use a UDF, the logic is easily ported into a cte or sub-query.
Select RetVal1
,RetVaL2
,y = sum(case when y is null then 0 else 1 end)
From [dbo].[udf-Range-Number-Span](0,100000,10000) A
Left Join Data B on y>=RetVal1 and y<RetVal2
Group By RetVal1,RetVal2
Returns
RetVal1 RetVaL2 y
0.00 10000.00 1
10000.00 20000.00 0
20000.00 30000.00 0
30000.00 40000.00 0
40000.00 50000.00 0
50000.00 60000.00 2
60000.00 70000.00 0
70000.00 80000.00 0
80000.00 90000.00 0
90000.00 100000.00 1
The UDF if needed
CREATE FUNCTION [dbo].[udf-Range-Number-Span] (#R1 money,#R2 money,#Incr money)
Returns Table
Return (
with cte0(M) As (Select cast((#R2-#R1)/#Incr as int)),
cte1(N) As (Select 1 From (Values(1),(1),(1),(1),(1),(1),(1),(1),(1),(1)) N(N)),
cte2(N) As (Select Top (Select M from cte0) Row_Number() over (Order By (Select NULL)) From cte1 a,cte1 b,cte1 c,cte1 d,cte1 e,cte1 f,cte1 g,cte1 h )
Select RetSeq=1,RetVal1=#R1,RetVal2=#R1+#Incr
Union All
Select N+1,(N*#Incr)+#R1,((N*#Incr)+#R1)+#Incr
From cte2,cte0
Where N<cte0.M
)
--Max 100 million observations
--Select * from [dbo].[udf-Range-Number-Span](1,4,.5)

MS SQL 2014 - Count of repeating ocurrences over range of dates

I have a table that tracks the Datetime of Incident_IDs created for specific Device_IDs and I am trying to find a way to track chronic issues over a range of dates. The definition of chronic issue is any Device_ID that had 3 or more Incident_IDs created in the past 5 days. I need to be able to search over a range of different dates (mostly monthly).
Given table:
IF OBJECT_ID('tempdb.dbo.#temp') IS NOT NULL
DROP TABLE #temp
CREATE TABLE #temp
(Device_ID INT,
Incident_ID INT,
Incident_Datetime DATETIME)
INSERT INTO #temp
VALUES
(2,1001,'2016-02-01'),
(3,1002,'2016-02-02'),
(2,1003,'2016-02-09'),
(2,1004,'2016-02-10'),
(5,1005,'2016-02-12'),
(2,1006,'2016-02-13'),
(5,1007,'2016-02-14'),
(5,1008,'2016-02-15'),
(3,1009,'2016-02-18'),
(3,1010,'2016-02-19'),
(3,1011,'2016-02-20'),
(5,1012,'2016-02-21'),
(3,1013,'2016-03-18'),
(3,1014,'2016-03-19'),
(3,1015,'2016-03-20');
The desired result for chronic issues for 02-2016 is:
Device_ID Incident_ID Incident_Datetime
2 1003 2/9/16 0:00
2 1004 2/10/16 0:00
2 1006 2/13/16 0:00
3 1009 2/18/16 0:00
3 1010 2/19/16 0:00
3 1011 2/20/16 0:00
5 1005 2/12/16 0:00
5 1007 2/14/16 0:00
5 1008 2/15/16 0:00
I have tried the following query which shows me the ascending count of incidents and allows me to find those device_ids that have had chronic issues but I'm having a hard time isolating all the incidents that make up the chronic issue while excluding those outliers that occurred outside the 3 day range.
SELECT c.Device_ID, c.Incident_ID, c.Incident_Datetime,
(SELECT COUNT(*)
FROM #temp AS t
WHERE
c.Device_ID = t.Device_ID
AND
t.Incident_Datetime BETWEEN DATEADD(DAY,-5,c.Incident_Datetime) AND c.Incident_Datetime) AS Incident_Count
FROM #temp AS c
WHERE
c.Incident_Datetime >= '2016-02-01'
AND
c.Incident_Datetime < '2016-03-01'
ORDER BY
Device_ID, Incident_Datetime
This is probably not quite as nice as Jake's answer, but here's an alternative solution that might work:
WITH cte AS
(
SELECT tmp.Device_ID, tmp.Incident_Datetime FROM #temp AS tmp
CROSS APPLY
(
SELECT Device_ID
FROM #temp AS t
WHERE tmp.Device_ID = t.Device_ID AND t.Incident_Datetime BETWEEN DATEADD(d,-5,tmp.Incident_Datetime) AND tmp.Incident_Datetime
GROUP BY Device_ID HAVING COUNT(Incident_ID) >= 3
) p
WHERE tmp.Incident_Datetime BETWEEN '02-01-2016' AND '03-01-2016'
)
SELECT f.*
FROM #temp f
INNER JOIN cte
ON f.Device_ID = cte.Device_ID
WHERE f.Incident_Datetime BETWEEN DATEADD(d,-5,cte.Incident_Datetime) AND cte.Incident_Datetime
GROUP BY f.Device_ID, f.Incident_ID, f.Incident_Datetime
ORDER BY f.Device_ID, f.Incident_Datetime
How about this...
DECLARE #StartDate datetime, #EndDate datetime
SET #StartDate='2016-02-01'
SET #EndDate='2016-03-01'
SELECT c.Device_ID, c.Incident_ID, c.Incident_DateTime FROM #temp c
INNER JOIN (SELECT t.Device_ID, Count(*) FROM #temp
WHERE t.Incident_DateTime BETWEEN DATEADD(dd, -3, c.Incident_DateTime) AND DATEADD(dd, +3, c.Incident_DateTime)
GROUP BY t.Device_ID
HAVING Count(*) > 2)) t ON c.Device_ID = t.Device_ID
AND c.Incident_DateTime BETWEEN #StartDate AND #EndDate
ORDER BY
c.Device_ID, c.Incident_Datetime
Here's a way to derive a running incidents within n days total:
with
incidents as (
select * from #temp cross apply (
select incident_datetime, 1 union all
select incident_datetime + 5, -1) x(dt, delta)),
rolling as (
select *, incidents_in_range = sum(delta)
over (partition by device_id order by dt)
from incidents)
select t.* from #temp t join rolling r
on r.device_id=t.device_id
and t.incident_datetime between r.incident_datetime - 5 and r.incident_datetime
where r.incidents_in_range >= 3
..basically find the points at which "3 incidents in 5 days" was reached, and then join back to include the incidents within 5 days.

How to get a derive 'N' Date Rows from a single record with From / To date columns?

Title sounds confusing but let me please explain:
I have a table that has two columns that provide a date range, and one column that provides a value. I need to query that table and "detail" the data such as this
Is it possible to do only using TSQL?
Additional Info
The table in question is about 2-3million records long (and growing)
Assuming the range of dates is fairly narrow, an alternative is to use a recursive CTE to create a list of all dates in the range and then join interpolate to it:
WITH LastDay AS
(
SELECT MAX(Date_To) AS MaxDate
FROM MyTable
),
Days AS
(
SELECT MIN(Date_From) AS TheDate
FROM MyTable
UNION ALL
SELECT DATEADD(d, 1, TheDate) AS TheDate
FROM Days CROSS JOIN LastDay
WHERE TheDate <= LastDay.MaxDate
)
SELECT mt.Item_ID, mt.Cost_Of_Item, d.TheDate
FROM MyTable mt
INNER JOIN Days d
ON d.TheDate BETWEEN mt.Date_From AND mt.Date_To;
I've also assumed an that date from and date to represent an inclusive range (i.e. includes both edges) - it is unusual to use inclusive BETWEEN on dates.
SqlFiddle here
Edit
The default MAXRECURSION on a recursive CTE in Sql Server is 100, which will limit the date range in the query to a span of 100 days. You can adjust this to a maximum of 32767.
Also, if you are filtering just a smaller range of dates in your large table, you can adjust the CTE to limit the number of days in the range:
WITH DateRange AS
(
SELECT CAST('2014-01-01' AS DATE) AS MinDate,
CAST('2014-02-16' AS DATE) AS MaxDate
),
Days AS
(
SELECT MinDate AS TheDate
FROM DateRange
UNION ALL
SELECT DATEADD(d, 1, TheDate) AS TheDate
FROM Days CROSS APPLY DateRange
WHERE TheDate <= DateRange.MaxDate
)
SELECT mt.Item_ID, mt.Cost_Of_Item, d.TheDate
FROM MyTable mt
INNER JOIN Days d
ON d.TheDate BETWEEN mt.Date_From AND mt.Date_To
OPTION (MAXRECURSION 0);
Update Fiddle
This can be achieved using Cursors.
I've simulated the test data provided and created another table with the name "DesiredTable" to store the data inside, and created the following cusror which achieved exactly what you are looking for:
SET NOCOUNT ON;
DECLARE #ITEM_ID int, #COST_OF_ITEM Money,
#DATE_FROM date, #DATE_TO date;
DECLARE #DateDiff INT; -- holds number of days between from & to columns
DECLARE #counter INT = 0; -- for loop counter
PRINT '-------- Begin the Date Expanding Cursor --------';
-- defining the cursor target statement
DECLARE Date_Expanding_Cursor CURSOR FOR
SELECT [ITEM_ID]
,[COST_OF_ITEM]
,[DATE_FROM]
,[DATE_TO]
FROM [dbo].[OriginalTable]
-- openning the cursor
OPEN Date_Expanding_Cursor
-- fetching next row data into the declared variables
FETCH NEXT FROM Date_Expanding_Cursor
INTO #ITEM_ID, #COST_OF_ITEM, #DATE_FROM, #DATE_TO
-- if next row is found
WHILE ##FETCH_STATUS = 0
BEGIN
-- calculate the number of days in between the date columns
SELECT #DateDiff = DATEDIFF(day,#DATE_FROM,#DATE_TO)
-- reset the counter to 0 for the next loop
set #counter = 0;
WHILE #counter <= #DateDiff
BEGIN
-- inserting rows inside the new table
insert into DesiredTable
Values (#COST_OF_ITEM, DATEADD(day,#counter,#DATE_FROM))
set #counter = #counter +1
END
-- fetching next row
FETCH NEXT FROM Date_Expanding_Cursor
INTO #ITEM_ID, #COST_OF_ITEM, #DATE_FROM, #DATE_TO
END
-- cleanup code
CLOSE Date_Expanding_Cursor;
DEALLOCATE Date_Expanding_Cursor;
The code fetches every row from your original table, then it calculates the number of days between DATE_FROM and DATE_TO columns, then using this number the script will create identical rows to be inserted inside the new table DesiredTable.
give it a try and let me know of the results.
You can generate an increment table and join it to your date From:
Query:
With inc(n) as (
Select ROW_NUMBER() over (order by (select 1)) -1 From (
Select 1 From (values(1), (1), (1), (1), (1), (1), (1), (1), (1), (1)) as x1(n)
Cross Join (values(1), (1), (1), (1), (1), (1), (1), (1), (1), (1)) as x2(n)
) as x(n)
)
Select item_id, cost, DATEADD(day, n, dateFrom), n From #dates d
Inner Join inc i on n <= DATEDIFF(day, dateFrom, dateTo)
Order by item_id
Output:
item_id cost Date n
1 100 2014-01-01 00:00:00.000 0
1 100 2014-01-02 00:00:00.000 1
1 100 2014-01-03 00:00:00.000 2
2 105 2014-01-08 00:00:00.000 2
2 105 2014-01-07 00:00:00.000 1
2 105 2014-01-06 00:00:00.000 0
2 105 2014-01-09 00:00:00.000 3
3 102 2014-02-14 00:00:00.000 3
3 102 2014-02-15 00:00:00.000 4
3 102 2014-02-16 00:00:00.000 5
3 102 2014-02-11 00:00:00.000 0
3 102 2014-02-12 00:00:00.000 1
3 102 2014-02-13 00:00:00.000 2
Sample Data:
declare #dates table(item_id int, cost int, dateFrom datetime, dateTo datetime);
insert into #dates(item_id, cost, dateFrom, dateTo) values
(1, 100, '20140101', '20140103')
, (2, 105, '20140106', '20140109')
, (3, 102, '20140211', '20140216');
Yet another way is to create and maintain calendar table, containing all dates for many years (in our app we have table for 30 years or so, extending every year). Then you can just link to calendar:
select <whatever you need>, calendar.day
from <your tables> inner join calendar on calendar.day between <min date> and <max date>
This approach allows to include additional information (holidays etc) in calendar table - sometimes very helpful.

Modifying the current row based on the previous row in sql server

I have a result set like this:
YearMonth Sales
201411 100
201412 100
201501 100
201502 100
201503 100
201504 100
201505 100
201506 100
201507 100
201508 100
Need to add another row with 4% more sales than the previous month. For example my Result should be
YearMonth Sales New Sales
201411 100 100.00
201412 100 104.00
201501 100 108.16
201502 100 112.49
201503 100 116.99
201504 100 121.67
201505 100 126.53
201506 100 131.59
201507 100 136.86
201508 100 142.33
Please help me to get the best way for it.
Got perfect answer for your requirement. It took long time to figure out. Just change the #Temp table name with your table name and verify the column names also.
DECLARE #nCurrentSale FLOAT
DECLARE #nYeatDate INT
DECLARE #nSale FLOAT
CREATE TABLE #TempNEW(YearMonth VARCHAR(10), Sales FLOAT, NewSale FLOAT)
SELECT TOP 1 #nCurrentSale = Sales FROM #Temp
ORDER BY (CAST('01/' + SUBSTRING (CAST(YearMonth AS VARCHAR), 5 , 2) + '/' + SUBSTRING (CAST(YearMonth AS
VARCHAR), 0 , 5) AS DATETIME)) ASC
DECLARE Cursor1 CURSOR FOR
SELECT YearMonth, Sales FROM #Temp
ORDER BY (CAST('01/' + SUBSTRING (CAST(YearMonth AS VARCHAR), 5 , 2) + '/' + SUBSTRING (CAST(YearMonth AS
VARCHAR), 0 , 5) AS DATETIME)) ASC
OPEN Cursor1
FETCH NEXT FROM Cursor1 INTO #nYeatDate, #nSale
WHILE ##FETCH_STATUS = 0
BEGIN
INSERT INTO #TempNEW(YearMonth, Sales, NewSale) VALUES(#nYeatDate, #nSale, CAST(#nCurrentSale AS DECIMAL(12,2)))
SET #nCurrentSale = #nCurrentSale + ((#nCurrentSale/100) * 4)
FETCH NEXT FROM Cursor1 INTO #nYeatDate, #nSale
END
CLOSE Cursor1
DEALLOCATE Cursor1
SELECT * FROM #TempNEW
Notify me with your status.
Yes it possible. But first you have to alter the table and add the extra column NewSales then try with this link
https://dba.stackexchange.com/questions/34243/update-row-based-on-match-to-previous-row
i think you can done it through this link
Also sql server support some "Computed Columns in SQL Server with Persisted Values"
using that you can specify the formula what you want, then the new column value will automatically created according to your formula
Here's two thoughts... Not super clear if I understood the use case... Also this solution will only work for SQL 2012 and up
So given the table
CREATE TABLE [dbo].[LagExample](
[YearMonth] [nvarchar](100) NOT NULL,
[Sales] [money] NOT NULL
)
First one is fairly simple and just assumes you are wanting to base the magnitude of your percentage increase on how many days came before it...
;WITH cte
as
(
SELECT YearMonth,
ROW_NUMBER() OVER (ORDER BY YearMonth) - 1 AS SalesEntry,
cast(LAG(Sales, 1,Sales) OVER (ORDER BY YearMonth) as float) as Sales
FROM LagExample
)
SELECT YearMonth,
Sales,
cast(Sales * POWER(cast(1.04 as float), SalesEntry) AS decimal(10,2)) as NewSales
FROM cte
The Second one uses a recursive CTE to calculate the value as you move along the months..
Here's a good link about recursive CTEs
http://www.codeproject.com/Articles/683011/How-to-use-recursive-CTE-calls-in-T-SQL
;with data
as
(
SELECT Lead(le.YearMonth, 1, null) OVER (ORDER BY le.YearMonth) as NextYearMonth,
cast(le.Sales as Decimal(10,4)) as Sales,
le.YearMonth
FROM LagExample le
)
,cte
as
(
SELECT *
FROM data
Where YearMonth = '201411'
UNION ALL
SELECT
data.NextYearMonth,
cast(cte.Sales * 1.04 as Decimal(10,4)) as Sales,
data.YearMonth
From cte join
data on data.YearMonth = cte.NextYearMonth
)
SELECT YearMonth, cast(Sales as Decimal(10,2))
FROM cte
order by YearMonth

Resources