Comparing date in subquery prevents SQL Server from using index - sql-server

I'm trying to filter only events that happened within 30 days after the company was created (RegisteredUtc).
Does anyone know how to rewrite this query to make sure that predicate uses the index that exists?
I've tried both with an index that has (eventtype, timeutc) and includes (companyid) and one that has (eventtype) and includes (companyid, timeutc).
This is the query that does what I want. NOTE: between these 3 queries it's only the last predicate in the first where clause that changes.
select
DATEPART(year, q.RegisteredUtc) as [year],
DATEPART(month, q.RegisteredUtc) as [month],
Count(*) as [Count]
from
(select
c.Id,
c.RegisteredUtc,
(select count(*)
from dbo.events sc
where sc.companyId = c.Id
and sc.eventtype = 'CreateInvoice'
and sc.TimeUtc < DATEADD(day, 30, c.RegisteredUtc)) as [Count]
from
dbo.companies c) as q
where
q.Count > 0
group by
DATEPART(year, q.RegisteredUtc), DATEPART(month, q.RegisteredUtc)
But it's very slow because of the and TimeUtc < DATEADD(day, 30, c.RegisteredUtc). Without that it uses the indexes and runs very fast but with that predicate there it does a Eager spool that is very expensive instead of using the index.
There are actually indexes available. Confirmed by simply swapping DATEADD(day, 30, c.RegisteredUtc)with a constant which made the query fast again.
This is also slow.
select
DATEPART(year, q.RegisteredUtc) as [year],
DATEPART(month, q.RegisteredUtc) as [month],
Count(*) as [Count]
from
(select
c.Id,
c.RegisteredUtc,
(select count(*)
from dbo.events sc
where sc.companyId = c.Id
and sc.eventtype = 'CreateInvoice'
and sc.TimeUtc < c.RegisteredUtc) as [Count]
from
dbo.companies c) as q
where
q.Count > 0
group by
DATEPART(year, q.RegisteredUtc), DATEPART(month, q.RegisteredUtc)
This query hits the index and works fast though but of course doesn't produce the correct results.
select
DATEPART(year, q.RegisteredUtc) as [year],
DATEPART(month, q.RegisteredUtc) as [month],
Count(*) as [Count]
from
(select
c.Id,
c.RegisteredUtc,
(select count(*)
from dbo.events sc
where sc.companyId = c.Id
and sc.eventtype = 'CreateInvoice'
and sc.TimeUtc = c.RegisteredUtc) as [Count]
from
dbo.companies c) as q
where
q.Count > 0
group by
DATEPART(year, q.RegisteredUtc), DATEPART(month, q.RegisteredUtc)

As I see it you are not even using the inner count
If it matches the join then the count > 0
select DATEPART(year, q.RegisteredUtc) as [year],
DATEPART(month, q.RegisteredUtc) as [month],
Count(*) as [Count]
from ( select distinct c.Id, c.RegisteredUtc
from dbo.companies c
join dbo.events sc
on sc.companyId = c.Id
and eventtype = 'CreateInvoice'
and TimeUtc < DATEADD(day, 30, c.RegisteredUtc)
) as q
group by DATEPART(year, q.RegisteredUtc), DATEPART(month, q.RegisteredUtc)
where exists may work better. Your query is confusing to me. You get a count just to compare it to > 0 and you are worried about not using an in index?

It might be worthwhile to compute the DATEADD(day, 30, c.RegisteredUtc) in the select and then compare TimeUtc with the computed column, in other words:
select
c.Id,
c.RegisteredUtc,
DATEADD(day, 30, c.RegisteredUtc) as RegisteredUtc30
(
select count(*)
from dbo.events sc
where sc.companyId = c.Id
and sc.eventtype = 'CreateInvoice'
and sc.TimeUtc < RegisteredUtc30
from dbo.companies c

Related

Adding the total number of distinct customers to a cte table in Microsoft SQL Server

I am using Microsoft SQL Server and am trying to achieve the following
Date
Distinct Customers last 30Days
2020-12-01
20000
2020-12-02
23000
What I am trying to get is that between 2020-11-01 and 2020-12-01 I had 20000 distinct customers.
I have created a cte table with the List of Dates as can be seen below:
WITH listdate AS
(
SELECT CAST('2020-11-01' AS datetime) DateValue
UNION ALL
SELECT DateValue + 1
FROM listdate
WHERE DateValue + 1 < getdate()
)
SELECT
cast(DateValue as date) as DateValue
FROM listdate d
Now I am trying to join the customer and usage table with the list of dates table, however, I am not getting the correct end result. The following is what I have tried doing:
WITH listdate AS
(
SELECT CAST('2020-11-01' AS datetime) DateValue
UNION ALL
SELECT DateValue + 1
FROM listdate
WHERE DateValue + 1 < getdate()
)
SELECT
cast(DateValue as date) as DateValue
,count(distinct case when m.CallDate between dateadd(dd,-30,cast(d.datevalue as date)) and cast(d.datevalue as date) then m.Customerid end) as Distinct_CID
FROM listdate d
join Usage m on d.DateValue=m.CallDate
left join Customer c on c.CustomerID=m.Customer
where c.customertype = 'type A'
group by d.DateValue
OPTION (MAXRECURSION 0)
Can someone maybe suggest a different way of how to solve such a query?
Thanks
I would go for a lateral join to bring the count of distinct customers for the last 30 days:
with listdate as (
select cast('20201101' as date) as datevalue
union all
select dateadd(day, 1, datevalue) from listdate where datevalue < cast(getdate() as date)
)
select ld.datevalue, x.cnt
from listdate ld
cross apply (
select count(distinct c.customerid) as cnt
from usage u
inner join customer c on c.customerid = u.customerid
where
c.customertype = 'type A'
and c.calldate >= dateadd(day, -29, datevalue)
and c.calldate < dateadd(day, 1, datevalue)
) x
option (maxrecursion 0)
Note that I simplified the parts related to dates: this uses proper literal dates and date arithmetics in the recursive query; the where clause of the subquery implements what I understand as the last 30 days (today + the preceding 29 days), and properly handles the time portion of calldate, if any.

How to set date clause for prevous month in sql?

I want to get last values for prevous month (universal, not just for january :))
How can I set up where condition? Now it works well for current month, but i want result for prevous.
SELECT MAX(v.timestamp) AS Date,
MAX(v.value) AS Stanje,
v.tag_id, t.prik_sifr, t.tag_name
FROM dbo.tag_values AS v INNER JOIN
dbo.typ_tag AS t ON v.tag_id = t.id
WHERE (t.prik_sifr IS NOT NULL) AND (t.unit = 'M3') AND
(DATEPART(YEAR, v.timestamp) = DATEPART(YEAR, SYSDATETIME())) AND
(DATEPART(MONTH, v.timestamp)= DATEPART(MONTH, SYSDATETIME())) GROUP BY v.tag_id, t.prik_sifr, t.tag_name)
You need to subtruct one month from the current date:
SELECT MAX(v.timestamp) AS Date,
MAX(v.value) AS Stanje,
v.tag_id, t.prik_sifr, t.tag_name
FROM dbo.tag_values AS v INNER JOIN
dbo.typ_tag AS t ON v.tag_id = t.id
WHERE (t.prik_sifr IS NOT NULL) AND (t.unit = 'M3') AND
(DATEPART(YEAR, v.timestamp) = DATEPART(YEAR, DATEADD(MONTH, -1 SYSDATETIME()))) AND
(DATEPART(MONTH, v.timestamp)= DATEPART(MONTH, DATEADD(MONTH, -1 SYSDATETIME())))
GROUP BY v.tag_id, t.prik_sifr, t.tag_name)

How to get a list of months and year between two dates in SQL Server

I have to get the list of months and year in between my dates. Currently it only returns month and year for dates that has data associated with it.
for example my dates is between: '8'+'/1'+'/'+'2015' and DATEADD(mm, 15, '8'+'/1'+'/'+'2016'
It only prints out: May2016, June2016, July2016, Auguest2016, September2016
I want it to print out all of the months and year in between. Here is my sql queries:
select d.id_base as case_id,
c.C_LAST_ACTION AS Docketed,
c.C_CASE_TYPE AS caseType,
ct.C_NAME As caseName,
ct.C_DESCRIPTION AS caseNameDescription,
case when d.c_mod_decision_id is not null then '' else DATENAME(mm, d.c_issue_date) + DATENAME(yyyy, d.c_issue_date) end as display
from t_case_decision d JOIN T_CASE_INPUT c on c.id = d.id_base JOIN T_CASE_TYPE ct on C_CASE_TYPE = ct.id
where cast(d.c_issue_date AS date) BETWEEN '8'+'/1'+'/'+'2015' and DATEADD(mm, 15, '8'+'/1'+'/'+'2016')
First, create a numbers table
CREATE TABLE Numbers(N INT)
insert into Numbers(N)
select top 1000000 row_number() over(order by t1.number) as N
from master..spt_values t1
cross join master..spt_values t2
then use DATEADD to list dates between desired values, like this
declare #iniDate as date
set #iniDate='20150801'
select dateadd(MONTH,N,#iniDate) dates
from Numbers
where N<15 order by N
These returns dates from #iniDate up to 15 months later
EDIT: try this, I don't have sql right now
select datename(mm, dateadd(MONTH,N,#iniDate))+datename(yyyy ,dateadd(MONTH,N,#iniDate)) display
from ( select top 15row_number() over(order by t1.number) as N
from master..spt_values t1
cross join master..spt_values t2) numbers right join (
select d.id_base as case_id,
c.C_LAST_ACTION AS Docketed,
c.C_CASE_TYPE AS caseType,
ct.C_NAME As caseName,
ct.C_DESCRIPTION AS caseNameDescription,
case when d.c_mod_decision_id is not null then '' else DATENAME(mm, d.c_issue_date) + DATENAME(yyyy, d.c_issue_date) end as display
from t_case_decision d JOIN T_CASE_INPUT c on c.id = d.id_base JOIN T_CASE_TYPE ct on C_CASE_TYPE = ct.id
where cast(d.c_issue_date AS date) BETWEEN '8'+'/1'+'/'+'2015' and DATEADD(mm, 15, '8'+'/1'+'/'+'2016')
sql-server
) qq
on datename(mm, dateadd(MONTH,N,#iniDate))+datename(yyyy ,dateadd(MONTH,N,#iniDate)) = qq.display
where N<15 order by N
If I understand what you're trying to accomplish, a recursive CTE might help. Here's a quick example of what you can do. The CTE will expand out into a list of dates, which you can then use as the base for your query.
The contents of the TargetData CTE may need to be adjusted, as I don't have a complete picture of your data structure.
DECLARE #startDate DATE = '1/1/2015';
DECLARE #endDate DATE = '7/31/2016';
-- Recursive CTE to generate a list of months within the date range:
WITH Months AS (
SELECT CONVERT(DATE, DATEADD(D, -(DAY(#startDate)) + 1, #startDate)) [MonthDate]
UNION ALL
SELECT DATEADD(M, 1, MonthDate)
FROM Months
WHERE MonthDate <= DATEADD(M, -1, #endDate)
),
TargetData AS (
-- This is a slightly modified version of the original query:
select
d.id_base as case_id,
c.C_LAST_ACTION AS Docketed,
c.C_CASE_TYPE AS caseType,
ct.C_NAME As caseName,
ct.C_DESCRIPTION AS caseNameDescription,
case when d.c_mod_decision_id is not null then '' else DATENAME(mm, d.c_issue_date) + DATENAME(yyyy, d.c_issue_date) end as display,
-- Return the "MonthDate" so that it can be left joined to the Months table:
DATEADD(D, -(DAY(d.c_issue_date)) + 1, d.c_issue_date) [MonthDate]
from t_case_decision d JOIN T_CASE_INPUT c on c.id = d.id_base JOIN T_CASE_TYPE ct on C_CASE_TYPE = ct.id
where cast(d.c_issue_date AS date) BETWEEN #startDate AND #endDate
)
SELECT
m.MonthDate,
DATENAME(mm, m.MonthDate) + DATENAME(yyyy, m.MonthDate),
td.*
FROM Months m
LEFT JOIN TargetData td ON td.MonthDate = m.MonthDate;
You need to join on primary keys between tables, I haven't seen a between statement with that syntax. So I suggest trying the following:
SELECT d.id_base as case_id, c.C_LAST_ACTION AS 'Docketed',c.C_CASE_TYPE AScaseType,ct.C_NAME As 'caseName', ct.C_DESCRIPTION AS 'caseNameDescription'
,CASE
WHEN d.c_mod_decision_id is not null THEN '' AS 'null_val'
ELSE CONCAT(YEAR(d.c_issue_dateDATENAME), MONTH(d.c_issue_date))
END AS 'display'
FROM t_case_decision d INNER JOIN T_CASE_INPUT c on c.id = d.id_base
INNER JOIN T_CASE_TYPE ct on c.id = ct.id
WHERE CONVERT(DATE,d.c_issue_date) BETWEEN '08/01/2015'
AND '08/01/2016';
I hope this helps or points you in the right direction :)

Count periods and duration of sick within a rolling 12 Month

I'm fairly new to SQL and find this site to be a brilliant resource. I'm hoping for a little bit of help with a task I've been assigned.
Basically I need to find the number of periods of sickness a member of staff has taken in the last 12 month and the duration of each period of sickness.
I have a simple table that looks like this:
Agent Date Status
A 01/07/2015 SHIFT
A 02/07/2015 SHIFT
A 03/07/2015 SICK
A 04/07/2015 SHIFT
A 05/07/2015 SHIFT
A 06/07/2015 SHIFT
B 01/07/2015 SICK
B 02/07/2015 SICK
B 03/07/2015 SHIFT
B 04/07/2015 SHIFT
B 05/07/2015 SICK
B 06/07/2015 SICK
C 01/07/2015 SHIFT
C 02/07/2015 SHIFT
C 03/07/2015 SICK
C 04/07/2015 SICK
C 05/07/2015 SICK
C 06/07/2015 SHIFT
I'm hoping someone can help me find some code that would produce the following kind of output:
Agent Days
A 1
B 2
B 2
C 3
Any help would be greatly appreciated.
Cheers
For sql-server:
select count(1) from TableA
where Status = 'SICK' and Date >= CAST(DATEADD(MONTH, -12, CURRENT_TIMESTAMP) AS DATE)
group by Agent
Grouped by periods of sickness:
WITH Cte AS(
SELECT *,
DATEDIFF(dd, '12/30/1899', [Date]) as [number],
RN = DATEDIFF(dd, '12/30/1899', [Date]) - ROW_NUMBER()
OVER(PARTITION BY Agent ORDER BY DATEDIFF(dd, '12/30/1899', [Date]))
FROM TableA
WHERE
[Status] = 'SICK' and
[Date] >= CAST(DATEADD(MONTH, -12, CURRENT_TIMESTAMP) AS DATE)
)
,CteFinal AS(
SELECT
Agent,
startNumber = MIN(number),
endNumber = MAX(number)
FROM Cte
GROUP BY Agent, RN
)
select Agent, endNumber - startNumber +1 as [Days] from CteFinal
group by agent, startNumber, endNumber
Sqlfiddle
In addition, for case with days off in table TableB:
WITH Cte AS(
SELECT distinct *,
DATEDIFF(dd, '12/30/1899', [Date]) as [number],
RN = DATEDIFF(dd, '12/30/1899', [Date]) - ROW_NUMBER()
OVER(PARTITION BY Agent ORDER BY DATEDIFF(dd, '12/30/1899', [Date]))
FROM (select distinct * from TableA
union all
select distinct Agent, b.Date, 'DAY OFF' from TableA cross join TableB b) TableA
WHERE
[Status] IN ('SICK', 'DAY OFF') and
[Date] >= CAST(DATEADD(MONTH, -12, CURRENT_TIMESTAMP) AS DATE)
)
,CteFinal AS(
SELECT
Agent,
startNumber = MIN(number),
endNumber = MAX(number),
dayOffs = SUM(CASE WHEN [Status] = 'DAY OFF' THEN 1 ELSE 0 END)
FROM Cte
GROUP BY Agent, RN
)
select Agent, endNumber - startNumber +1 - sum(dayOffs) as [Days] from CteFinal
group by agent, startNumber, endNumber
having(endNumber - startNumber +1 - sum(dayOffs) > 0)
Sqlfiddle

How to query Open-high-low-close (OHLC) data from SQL Server

I'm trying to retrieve data for a Open-high-low-close (OHLC) chart directly from the database, it's the kind of chart you see of stocks. Is this possible, and if, how?
I have a table like this (simplified):
Date | Price | PriceType
A record is created for each day, I will report per month / year, not per day as used for stocks.
I would like to query something like this:
SELECT PriceType, MAX(Price) as High, MIN(Price) as Low, [Price of first item of month] as Open, [Price of last item of month] as Close GROUP BY PriceType, Year(Date), Month(Date)
To access the SQL Server I use LLBLGen, so an anwser based on that technology would be great, a generic SQL server will do too!
It's SQL 2005, but 2008 is also an option.
Thanks.
This appears to work. There may well be a less verbose way to do it.
--create test data
CREATE TABLE #t
(priceDate DATETIME
,price MONEY
,priceType CHAR(1)
)
INSERT #t
SELECT '20090101',100,'A'
UNION SELECT '20090102',500,'A'
UNION SELECT '20090103',20 ,'A'
UNION SELECT '20090104',25 ,'A'
UNION SELECT '20090105',28 ,'A'
UNION SELECT '20090131',150,'A'
UNION SELECT '20090201',501,'A'
UNION SELECT '20090203',21 ,'A'
UNION SELECT '20090204',26 ,'A'
UNION SELECT '20090205',29 ,'A'
UNION SELECT '20090228',151,'A'
UNION SELECT '20090101',100,'B'
UNION SELECT '20090102',500,'B'
UNION SELECT '20090103',20 ,'B'
UNION SELECT '20090104',25 ,'B'
UNION SELECT '20090105',28 ,'B'
UNION SELECT '20090131',150,'B'
UNION SELECT '20090201',501,'B'
UNION SELECT '20090203',21 ,'B'
UNION SELECT '20090204',26 ,'B'
UNION SELECT '20090205',29 ,'B'
UNION SELECT '20090228',151,'B'
--query
;WITH rangeCTE
AS
(
SELECT MIN(priceDate) minDate
,MAX(priceDate) maxDate
FROM #t
)
,datelistCTE
AS
(
SELECT CAST(CONVERT(CHAR(6),minDate,112) + '01' AS DATETIME) AS monthStart
,DATEADD(mm,1,CAST(CONVERT(CHAR(6),minDate,112) + '01' AS DATETIME)) -1 AS monthEnd
,1 AS monthID
FROM rangeCTE
UNION ALL
SELECT DATEADD(mm,1,monthStart)
,DATEADD(mm,2,monthStart) - 1
,monthID + 1
FROM datelistCTE
WHERE monthStart <= (SELECT maxDate FROM rangeCTE)
)
,priceOrderCTE
AS
(
SELECT *
,ROW_NUMBER() OVER (PARTITION BY monthID, priceType
ORDER BY priceDate
) AS rn1
,ROW_NUMBER() OVER (PARTITION BY monthID, priceType
ORDER BY priceDate DESC
) AS rn2
,ROW_NUMBER() OVER (PARTITION BY monthID, priceType
ORDER BY price DESC
) AS rn3
,ROW_NUMBER() OVER (PARTITION BY monthID, priceType
ORDER BY price
) AS rn4
FROM datelistCTE AS d
JOIN #t AS t
ON t.priceDate BETWEEN d.monthStart AND d.monthEnd
WHERE monthStart <= (SELECT maxDate FROM rangeCTE)
)
SELECT o.MonthStart
,o.priceType
,o.Price AS opening
,c.price AS closing
,h.price AS high
,l.price AS low
FROM priceOrderCTE AS o
JOIN priceOrderCTE AS c
ON c.priceType = o.PriceType
AND c.monthID = o.MonthID
JOIN priceOrderCTE AS h
ON h.priceType = o.PriceType
AND h.monthID = o.MonthID
JOIN priceOrderCTE AS l
ON l.priceType = o.PriceType
AND l.monthID = o.MonthID
WHERE o.rn1 = 1
AND c.rn2 = 1
AND h.rn3 = 1
AND l.rn4 = 1
This is a little query I wrote that seems to work nicely for one time span at a time. All you need to do is comment the select DATEPARTS in order to get to the timespan you are looking for. Or you could just make multiple views for different timespans. Also the underlying data table uses Bid Ask tick style data. If you are using mids or last prices you could eliminate the case statements from the selects.
Select
tmp.num,
rf.CurveName,
rf.Period as Period,
CASE WHEN (tmp2.Bid is null or tmp2.Ask is null) then isnull(tmp2.Bid,0)+isnull(tmp2.Ask,0) else (tmp2.Bid+tmp2.Ask)/2 end as [Open],
tmp.Hi,
tmp.Lo,
CASE WHEN (rf.Bid is null or Rf.Ask is null) then isnull(rf.Bid,0)+isnull(rf.Ask,0) else (rf.Bid+rf.Ask)/2 end as [Close],
tmp.OpenDate,
tmp.CloseDate,
tmp.yr,
tmp.mth,
tmp.wk,
tmp.dy,
tmp.hr
from BidAsk rf inner join
(SELECT count(CurveName)as num,CurveName,
Period,
max(CASE WHEN (Bid is null or Ask is null) then isnull(Bid,0)+isnull(Ask,0) else (Bid+Ask)/2 end) as Hi,
min(CASE WHEN (Bid is null or Ask is null) then isnull(Bid,0)+isnull(Ask,0) else (Bid+Ask)/2 end) as Lo,
max(CurveDateTime) as CloseDate, min(CurveDateTime) as OpenDate,
DATEPART(year, CurveDateTime) As yr,
DATEPART(month, CurveDateTime) As mth,
DATEPART(week, CurveDateTime) As wk,
DATEPART(Day, CurveDateTime) as dy,
DATEPART(Hour, CurveDateTime) as hr
--DATEPART(minute, CurveDateTime) as mnt
FROM
BidAsk
GROUP BY
CurveName,Period,
DATEPART(year, CurveDateTime),
DATEPART(month, CurveDateTime),
DATEPART(week, CurveDateTime),
DATEPART(Day, CurveDateTime) ,
DATEPART(Hour, CurveDateTime)
--DATEPART(minute, CurveDateTime)
) tmp on
tmp.CurveName=rf.CurveName and
tmp.CloseDate=rf.CurveDateTime and
tmp.Period=rf.Period
inner join BidAsk tmp2 on
tmp2.CurveName=rf.CurveName and
tmp2.CurveDateTime=tmp.Opendate and
tmp2.Period=rf.Period
ORDER BY
CurveName,Period,tmp.yr,tmp.mth
--DATEPART(year, CurveDateTime),
--DATEPART(month, CurveDateTime)
--DATEPART(day, CurveDateTime),
--DATEPART(Hour, CurveDateTime),
--DATEPART(minute, CurveDateTime) )

Resources