Amazon Redshift MAX_STEPS limitation - database

When I try to run a huge query over Amazon Redshift I get the following error:
"m_num_steps < MAX_STEPS - exceeded maximum plan steps of 100"
There are 4 tables at Redshift. The fact table has about 400 million rows. The query is a "WITH" clause with multiple joins, agregations, analytic and window functions.
The question is: there is a way to avoid this "MAX_STEPS = 100" limitation?
The query (simplified):
WITH
-- Regr
a27 as (
select id_uc, case when count(*) = 0 or VAR_POP(numero_de_meses) = 0 then null else trunc(((SUM(consumo_normalizado * numero_de_meses) - SUM(numero_de_meses) * SUM(consumo_normalizado) / count(*)) / count(*)) / VAR_POP(numero_de_meses), 2) end N2542, case when count(*) = 0 or STDDEV_POP(consumo_normalizado) = 0 or STDDEV_POP(numero_de_meses) = 0 then null else trunc(POWER(((SUM(consumo_normalizado * numero_de_meses) - SUM(numero_de_meses) * SUM(consumo_normalizado) / count(*)) / count(*)) / (STDDEV_POP(consumo_normalizado) * STDDEV_POP(numero_de_meses)), 2), 2) end N2554 from (
select id_uc,
rank() over (partition by canal.id_uc order by canal.data_referencia) numero_de_meses,
case
when stddev(consumo_faturado) over (partition by canal.id_uc) <> 0
then (consumo_faturado - avg(consumo_faturado) over (partition by canal.id_uc)) / stddev(consumo_faturado) over (partition by canal.id_uc)
else null end as consumo_normalizado
from fato_uc canal
where exists (select 1 from eventos_resultado_atual evt where canal.id_uc = evt.id_uc and canal.data_referencia between E_Fat_12_meses_atras and ultimo_faturamento)
) group by id_uc)
-- Regr
,a28 as (
select id_uc, case when count(*) = 0 or VAR_POP(numero_de_meses) = 0 then null else case when count(*) = 0 or VAR_POP(numero_de_meses) = 0 then null else trunc(((SUM(consumo_normalizado * numero_de_meses) - SUM(numero_de_meses) * SUM(consumo_normalizado) / count(*)) / count(*)) / VAR_POP(numero_de_meses), 2) end end N2545, case when count(*) = 0 or STDDEV_POP(consumo_normalizado) = 0 or STDDEV_POP(numero_de_meses) = 0 then null else trunc(POWER(((SUM(consumo_normalizado * numero_de_meses) - SUM(numero_de_meses) * SUM(consumo_normalizado) / count(*)) / count(*)) / (STDDEV_POP(consumo_normalizado) * STDDEV_POP(numero_de_meses)), 2), 2) end N2557 from (
select id_uc,
rank() over (partition by canal.id_uc order by canal.data_referencia) numero_de_meses,
case
when stddev(consumo_faturado) over (partition by canal.id_uc) <> 0
then (consumo_faturado - avg(consumo_faturado) over (partition by canal.id_uc)) / stddev(consumo_faturado) over (partition by canal.id_uc)
else null end as consumo_normalizado
from fato_uc canal
where exists (select 1 from eventos_resultado_atual evt where canal.id_uc = evt.id_uc and canal.data_referencia between E_Fat_12_meses_atras and E_2457)
) group by id_uc)
-- Corr
,a65 as (
select evt.id_uc, case when count(*) = 0 or STDDEV_POP(c1.avg_cons) = 0 or STDDEV_POP(c2.consumo_faturado) = 0 then null else trunc(((SUM(c1.avg_cons * c2.consumo_faturado) - SUM(c2.consumo_faturado) * SUM(c1.avg_cons) / count(*)) / count(*)) / (STDDEV_POP(c1.avg_cons) * STDDEV_POP(c2.consumo_faturado)), 2) end N2720
from eventos_resultado_atual evt inner join VIZ_FAT c1 on evt.id_uc = c1.id_uc
inner join fato_uc c2 on evt.id_uc = c2.id_uc and date_trunc('month', c1.mes_ref) = date_trunc('month', c2.data_referencia)
where c1.mes_ref between E_Fat_12_meses_atras and ultimo_faturamento
and c2.data_referencia between E_Fat_12_meses_atras and ultimo_faturamento
group by evt.id_uc)
-- Corr
,a66 as (
select evt.id_uc, case when count(*) = 0 or STDDEV_POP(c1.avg_cons) = 0 or STDDEV_POP(c2.consumo_faturado) = 0 then null else trunc(((SUM(c1.avg_cons * c2.consumo_faturado) - SUM(c2.consumo_faturado) * SUM(c1.avg_cons) / count(*)) / count(*)) / (STDDEV_POP(c1.avg_cons) * STDDEV_POP(c2.consumo_faturado)), 2) end N2723
from eventos_resultado_atual evt inner join VIZ_FAT c1 on evt.id_uc = c1.id_uc
inner join fato_uc c2 on evt.id_uc = c2.id_uc and date_trunc('month', c1.mes_ref) = date_trunc('month', c2.data_referencia)
where c1.mes_ref between E_Fat_12_meses_atras and E_2457
and c2.data_referencia between E_Fat_12_meses_atras and E_2457
group by evt.id_uc)
SELECT *
FROM eventos_resultado_atual
left join a27 on eventos_resultado_atual.id_uc = a27.id_uc
left join a28 on eventos_resultado_atual.id_uc = a28.id_uc
left join a65 on eventos_resultado_atual.id_uc = a65.id_uc
left join a66 on eventos_resultado_atual.id_uc = a66.id_uc

Your problem is with redshift and as always amazon have no solution,try breaking the query in little tasks, be careful with working with lot of data in redshift have serious issues with big data

My recommendation is to break your query into a few sub queries and insert the results into tables along the process. Please share query and I will help you break it up into a few queries that will run more efficient. If you can also share the redshift provided explain plain I I can go over how to use that to do what I would do...

Related

Issue with >= in the WHERE clause

I'm at my whit's end with this one, frustrated because I'm sure it's something simple and I need a second set of eyes. I've tried this several different ways and I continue to run into an issue where the query will run for a full 24 hours with no results...
I've narrowed the issue down to this section of the query:
WHERE sub.[Savings (%)] >= 10
I ran into so many issues that I ended up nesting the entire query into a sub query, trying to take most of my calculations out of the "Where" clause, but to no effect.
This is all part of a much larger query, so I'll post the abbreviated portion that is relevant.
Here's the very beginning of the query:
SELECT *
FROM
(
SELECT
LD.Region
,DM.ShortName AS DM
,D.Lcode
,LD.locationname
,D.UnitID
,D.dPlaced
,D.dCancelled
,D.sRentalType
,D.[Quoted Rate]
,Calcs.MinStdRate AS [Current Rate]
,-(Calcs.MinStdRate - D.[Quoted Rate]) AS [Savings ($)]
,((-(Calcs.MinStdRate - D.[Quoted Rate]))/NULLIF(D.[Quoted Rate],0))*100 AS [Savings (%)]
,D.sPlanName
,Calcs.[# Vacant]
FROM...
Then this is the WHERE clause at the very end of the query that causes the issue. The entire query beneath "SELECT * FROM" (shown up above) is called "Sub":
) AS sub
WHERE sub.[Savings (%)] >= 10
AND sub.[Current Rate] <> 0
AND sub.[Quoted Rate] <> 0
When I notate "sub.[Savings (%)] >= 10 AND" out of the query, the whole thing runs in about 5 seconds. With it, it runs for hours on end and never produces results...
What am I missing?
Updating to show whole query per request:
SELECT *
FROM
(
SELECT
LD.Region
,DM.ShortName AS DM
,D.Lcode
,LD.locationname
,D.UnitID
,D.dPlaced
,D.dCancelled
,D.sRentalType
,D.[Quoted Rate]
,Calcs.MinStdRate AS [Current Rate]
,-(Calcs.MinStdRate - D.[Quoted Rate]) AS [Savings ($)]
,CAST(((-(Calcs.MinStdRate - D.[Quoted Rate]))/NULLIF(D.[Quoted Rate],0))*100 AS DECIMAL (18,2)) AS [Savings (%)]
,D.sPlanName
,Calcs.[# Vacant]
FROM
(SELECT
s.sLocationCode AS lcode
,w.dPlaced
,L.dLease
,u.UnitID
,CASE
WHEN w.dCancelled IS NOT NULL THEN w.dCancelled
WHEN (w.dCancelled IS NULL AND w.dExpires <GETDATE()) THEN w.dExpires
ELSE NULL
END AS dCancelled
,CASE
WHEN (w.dCancelled IS NOT NULL OR w.dExpires <GETDATE()) THEN 'Lost'
WHEN w.QTRentalTypeID = 3 THEN 'Rented'
WHEN (w.QTRentalTypeID = 2 OR w.QTRentalTypeID = 1) THEN 'Active'
END AS sRentalType
,w.dcRate_Quoted AS 'Quoted Rate'
,c.sPlanName
,CONCAT(U.dcWidth,'x',U.dcLength) AS sSize
,UT.sTypeName
FROM CompanyDBs.dbo.waitings AS W
LEFT OUTER JOIN CompanyDBs.dbo.Ledgers AS L
ON W.LedgerID = L.LedgerID
JOIN CompanyDBs.dbo.sites AS S
ON W.SiteID = S.SiteID
JOIN CompanyDBs.dbo.units AS U
ON W.UnitID1 = U.UnitID
JOIN CompanyDBs.dbo.UnitTypes AS UT
ON U.UnitTypeID = UT.UnitTypeID
JOIN CompanyDBs.dbo.ConcessionPlans AS C
ON W.ConcessionID = C.ConcessionID
WHERE W.dCancelled < GETDATE() AND W.dCancelled >= DATEADD(DD,-60,CAST(GETDATE() AS DATE))
UNION ALL
SELECT
s.sLocationCode AS lcode
,w.dPlaced
,L.dLease
,u.UnitID
,CASE
WHEN w.dCancelled IS NOT NULL THEN w.dCancelled
WHEN (w.dCancelled IS NULL AND w.dExpires <GETDATE()) THEN w.dExpires
ELSE NULL
END AS dCancelled
,CASE
WHEN (w.dCancelled IS NOT NULL OR w.dExpires <GETDATE()) THEN 'Lost'
WHEN w.QTRentalTypeID = 3 THEN 'Rented'
WHEN (w.QTRentalTypeID = 2 OR w.QTRentalTypeID = 1) THEN 'Active'
END AS sRentalType
,w.dcRate_Quoted AS 'Quoted Rate'
,c.sPlanName
,CONCAT(U.dcWidth,'x',U.dcLength) AS sSize
,UT.sTypeName
FROM CompanyDBs1.dbo.waitings AS W
LEFT OUTER JOIN CompanyDBs1.dbo.Ledgers AS L
ON W.LedgerID = L.LedgerID
JOIN CompanyDBs1.dbo.sites AS S
ON W.SiteID = S.SiteID
JOIN CompanyDBs1.dbo.units AS U
ON W.UnitID1 = U.UnitID
JOIN CompanyDBs1.dbo.UnitTypes AS UT
ON U.UnitTypeID = UT.UnitTypeID
JOIN CompanyDBs1.dbo.ConcessionPlans AS C
ON W.ConcessionID = C.ConcessionID
WHERE W.dCancelled < GETDATE() AND W.dCancelled >= DATEADD(DD,-60,CAST(GETDATE() AS DATE))) AS D
LEFT OUTER JOIN
(SELECT
P.SiteID
,P.sLocationCode
,P.UnitID
,p.UnitCODE
,groupeddata.[# Vacant]
,GroupedData.MinStdRate
FROM
(SELECT
S2.sLocationCode
,S2.SiteID
,R2.UnitID
,CONCAT(S2.sLocationCode
,'-'
,S2.sSiteName
,' '
,R2.stypename
,' '
,CASE WHEN R2.iFloor > 1 THEN 'Up ' WHEN R2.ifloor < 1 THEN 'Down ' WHEN R2.ifloor = 1 THEN '1 ' END
,CASE WHEN R2.bPower = 1 THEN 'Power ' ELSE 'No Power ' END
,CASE WHEN R2.bClimate = 1 THEN 'CC ' ELSE 'No CC ' END
,CASE WHEN R2.bInside = 1 THEN 'In ' ELSE 'Out ' END
,CASE WHEN R2.bAlarm = 1 THEN 'Alarm ' ELSE 'No Alarm ' END
,CAST(R2.dcWidth AS FLOAT)
,'x'
,CAST(R2.dcLength AS FLOAT)) AS UnitCODE
FROM Operations.dbo.RentRoll AS R2
JOIN
(
SELECT *
FROM CompanyDBs.dbo.sites
UNION ALL
SELECT *
FROM CompanyDBs1.dbo.sites
) AS S2
ON R2.siteid = S2.SiteID
WHERE
R2.ddeleted is NULL
AND R2.bRentable = 1
AND R2.brented = 0 ) AS P
JOIN
(
SELECT
S.SiteID
,s.sLocationCode
,UnitCode.UnitCODE
,SUM(CASE
WHEN R.brented = 1 THEN 0
ELSE 1
END) AS [# Vacant]
,MIN(R.dcStdRate) AS MinStdRate
FROM Operations.dbo.RentRoll AS R
JOIN
(
SELECT *
FROM CompanyDBs.dbo.sites
UNION ALL
SELECT *
FROM CompanyDBs1.dbo.sites
) AS S
ON R.siteid = S.SiteID
JOIN
(
SELECT
S1.SiteID
,UnitID
,CONCAT(S1.sLocationCode
,'-'
,S1.sSiteName
,' '
,R1.stypename
,' '
,CASE WHEN R1.iFloor > 1 THEN 'Up ' WHEN R1.ifloor < 1 THEN 'Down ' WHEN R1.ifloor = 1 THEN '1 ' END
,CASE WHEN R1.bPower = 1 THEN 'Power ' ELSE 'No Power ' END
,CASE WHEN R1.bClimate = 1 THEN 'CC ' ELSE 'No CC ' END
,CASE WHEN R1.bInside = 1 THEN 'In ' ELSE 'Out ' END
,CASE WHEN R1.bAlarm = 1 THEN 'Alarm ' ELSE 'No Alarm ' END
,CAST(R1.dcWidth AS FLOAT)
,'x'
,CAST(R1.dcLength AS FLOAT)) AS UnitCODE
FROM Operations.dbo.RentRoll AS R1
JOIN
(
SELECT *
FROM CompanyDBs.dbo.sites
UNION ALL
SELECT *
FROM CompanyDBs1.dbo.sites
) AS S1
ON R1.siteid = S1.SiteID
) AS UnitCode
ON CONCAT(R.siteid, R.unitid) = CONCAT(UnitCode.siteid,UnitCode.UnitID)
WHERE
s.sLocationCode <> 'L003'
AND s.sLocationCode <> 'L021'
AND s.sLocationCode <> 'LSETUP'
AND s.sLocationCode <> 'LTRAIN'
AND R.bRented = 0
GROUP BY s.siteid, s.slocationcode, UnitCode.UnitCODE
) AS GroupedData
ON P.UnitCODE = GroupedData.UnitCODE
) AS Calcs
ON CONCAT(D.lcode,D.unitid) = CONCAT(Calcs.slocationcode,Calcs.unitid)
JOIN operations.dbo.westport_locationdata AS LD
ON D.lcode = ld.lcode
JOIN operations.dbo.Westport_DMs AS DM
ON LD.DMID = DM.DMID
) AS sub
WHERE sub.[Savings (%)] >= CAST(10.0 AS DECIMAL (18,2))
AND sub.[Current Rate] <> 0
AND sub.[Quoted Rate] <> 0
Unfortunately you are posting not the whole query. I could image that some indexes are missing.
I give you the strong advice to run your query in SQL Server Management Studio with the option "Query" -> "Include Actual Execution Plan" checked.
With this, SSMS will execute the query and tell you afterwards if indexes are missing and how much you could improve by setting them. You will get also a picture oh how the query is running.
Cast your where clause filter value WHERE sub.[Savings (%)] >= cast(10.0 as decimal(18,2)) so there isn't an implicit conversion from int to decimal.
https://learn.microsoft.com/en-us/sql/t-sql/data-types/data-type-conversion-database-engine?view=sql-server-ver15#:~:text=Implicit%20conversions%20are%20not%20visible,converts%20to%20date%20style%200.

How to get a windowed function working in WHERE clause

I know that there are quite many threads on "Windowed functions can only appear in the select or ORDER BY clases" topic, but I have read them through and just don't seem to get any of those trick to work for me. So here I go.
My Query you can see below the line. I have highlighted the part of the Query which is "causing" me issues or problems. The thing is that I would like to get
"LocalCurrentAmount not between -1.000000 and 1.000000"
in somehow. I have tried the CTE versioon, but somehow didn't work and my declare tables and then the details view started causing problems.
Would really appreciate your help!
Jaanis
declare #exceptions1 table (CustomerCode varchar(7), Exception_comment varchar(15));
insert into #exceptions1 (CustomerCode, Exception_comment)
VALUES
(3514437,'Exception'),(3500977,'Exception'),(3295142,'Exception'), ...
declare #exceptions2 table (CustomerCode2 varchar(7), Exception_comment2 varchar(15));
insert into #exceptions2 (CustomerCode2, Exception_comment2)
VALUES
(3390437,'VIP SE') ,(3390438,'VIP SE') ,(3390481,'VIP SE'), ...
declare #exceptions3 table (CustomerCode3 varchar(7), Exception_comment3 varchar(15));
insert into #exceptions1 (CustomerCode, Exception_comment)
VALUES
(1530350, 'DK Exception'), (1533834, 'DK Exception'), (1530002, 'DK Exception'), ...
with Details as
(
select ard.AccountsReceivableHeaderID, sum(ard.TransactionAmountOC) as DetailAmountOC , Max(DetailSequenceCode) as DetailSequenceCode, Max( ard.BatchDate) as BatchDate
from dmbase.fAccountsReceivableDetail ard
join dmbase.fAccountsReceivable arh on ard.AccountsReceivableHeaderID = arh.AccountsReceivableID
where ard.BatchDate <= getdate()
group by AccountsReceivableHeaderID
)
SELECT
comp.CompanyCode
,convert(varchar(10),ar.InvoiceDate, 103) as Invoice_date -- dd/MM/yyyy format
,case
when ar.IsCreditMemo = 'Y' then 'Memo (Credit/Debit)'
when ar.InvoiceCode = '0' then 'Payment'
else 'Invoice'
end as Description
,isnull(cm.SummaryInvoiceCode, ar.InvoiceSummaryCode) InvoiceSummaryCode
,case
when len(ar.InvoiceSequenceCode) = '1' then CONCAT(ar.InvoiceCode,'-000',ar.InvoiceSequenceCode)
when len(ar.InvoiceSequenceCode) = '2' then CONCAT(ar.InvoiceCode,'-00',ar.InvoiceSequenceCode)
when len(ar.InvoiceSequenceCode) = '3' then CONCAT(ar.InvoiceCode,'-0',ar.InvoiceSequenceCode)
else CONCAT(ar.InvoiceCode,'-',ar.InvoiceSequenceCode)
end as Invoice#
,**(ar.OriginalInvoiceAmountOC
+
case
when row_number() over (partition by AccountsReceivableID order by ar.InvoiceCode) = 1 then isnull(vat.vatAdjustment, 0)
else 0
end
+
coalesce(det.DetailAmountOC, 0)) * coalesce(cer.CurrencyExchangeRate, ar.CurrencyExchangeRate) AS LocalCurrentAmount**
,(ar.OriginalInvoiceAmountOC
+
case
when row_number() over (partition by AccountsReceivableID order by ar.InvoiceCode) = 1 then isnull(vat.vatAdjustment, 0)
else 0
end
+ coalesce(det.DetailAmountOC, 0)) AS CurrentAmount
,ar.OriginalInvoiceAmountOC
+
case
when row_number() over (partition by AccountsReceivableID order by ar.InvoiceCode) = 1 then isnull(vat.vatAdjustment, 0)
else 0
end as OriginalInvoiceAmountOC
,ar.InvoiceCurrencyCode
,cust.CustomerCode
,upper(cust.CustomerName) as CustomerName
from
dmbase.fAccountsReceivable ar
INNER JOIN dmbase.dlocation loc
ON loc.LocationID = ar.LocationID
INNER JOIN dmbase.dCustomer cust
ON cust.CustomerID = ar.CustomerID
LEFT JOIN dmbase.VatAdjustment vat
on ar.InvoiceCode = vat.contractNumber
and ar.InvoiceSequenceCode = vat.invoiceSequence
and cust.CustomerCode = vat.CustomerNumber
and loc.CompanyCode = vat.companyCode
inner join dmbase.dCompany comp
on (ar.CompanyID = comp.CompanyID)
left join dmbase.dAccountsReceivableInvoiceStatus aris
on (aris.ARInvoiceStatusAMID=ar.ARInvoiceStatusAMID)
left hash join Details det
on (ar.AccountsReceivableID = det.AccountsReceivableHeaderID)
left join dmbase.dCurrencyExchangeRate cer
on (comp.CompanyCode = cer.CompanyCode
and ar.InvoiceCurrencyCode = cer.CurrencyCode
and case ar.InvoiceDate when '1900-01-01' then getdate() else ar.InvoiceDate end between cer.ValidFrom and cer.ValidTo)
left join dmbase.fContractClosedHeader ccd
on ccd.ContractNumber = ar.InvoiceCode
and ccd.ContractSeqNumber = ar.InvoiceSequenceCode
and ccd.CompanyID = ar.CompanyID
and ccd.ContractNumber!='0'
and ccd.CreditMemoContractNumber != '0'
left join dmbase.fAccountsReceivableHeader cm
on ccd.CreditMemoContractNumber = cm.ContractNumber
and ccd.CreditMemoSequenceCode = cm.ContractSeqNumber
and cm.CompanyID = ccd.CompanyID
where
(aris.ARInvoiceStatusCode = 'OP' or (ar.LastPaymentDate >= getdate())
or (ar.TotalAdjustmentsAmountOC <> 0 and ar.CurrentAmountLC = 0
and (ar.OriginalInvoiceAmountOC + isnull(vat.vatAdjustment, 0) ) + coalesce(det.DetailAmountOC, 0) <> 0)
)
and ar.OriginalInvoiceAmountOC <= 0
and ar.IsCreditMemo = 'Y' -- ainult Memo (Credit/Debit)
and cust.InternalCustomerType = 'External'
and cust.CustomerName not in ('RR AJM', 'RAMIRENT', 'Ramirent')
and cust.CustomerName not like '%[7][0-9][0-9][0-9]%'
and ar.InvoiceDate <= EOMONTH(getdate(),-3
and cust.CustomerCode NOT IN
(select CustomerCode from #exceptions1
union
select CustomerCode2 from #exceptions2
union
select CustomerCode3 from #exceptions3)
order by Invoice_date
When using a Window function, like you said in your post, you can't use it in the WHERE clause. The common solution, therefore, is to use a CTE and reference it in the WHERE outside of it. I've used your CTE, however, without any kind of sample data this is a total guess. I've also left a lot of comments for you, and changed some other parts of the SQL, as some of the clauses you have will effect your query's performance.
WITH
Details AS
(SELECT ard.AccountsReceivableHeaderID,
SUM(ard.TransactionAmountOC) AS DetailAmountOC,
MAX(DetailSequenceCode) AS DetailSequenceCode,
MAX(ard.BatchDate) AS BatchDate
FROM dmbase.fAccountsReceivableDetail ard
JOIN dmbase.fAccountsReceivable arh ON ard.AccountsReceivableHeaderID = arh.AccountsReceivableID
WHERE ard.BatchDate <= GETDATE()
GROUP BY AccountsReceivableHeaderID),
Summary AS(
SELECT comp.CompanyCode,
CONVERT(varchar(10), ar.InvoiceDate, 103) AS Invoice_date, -- dd/MM/yyyy format
CASE
WHEN ar.IsCreditMemo = 'Y' THEN 'Memo (Credit/Debit)'
WHEN ar.InvoiceCode = '0' THEN 'Payment'
ELSE 'Invoice'
END AS Description,
ISNULL(cm.SummaryInvoiceCode, ar.InvoiceSummaryCode) AS InvoiceSummaryCode,
CASE
WHEN LEN(ar.InvoiceSequenceCode) = '1' THEN CONCAT(ar.InvoiceCode, '-000', ar.InvoiceSequenceCode)
WHEN LEN(ar.InvoiceSequenceCode) = '2' THEN CONCAT(ar.InvoiceCode, '-00', ar.InvoiceSequenceCode)
WHEN LEN(ar.InvoiceSequenceCode) = '3' THEN CONCAT(ar.InvoiceCode, '-0', ar.InvoiceSequenceCode)
ELSE CONCAT(ar.InvoiceCode, '-', ar.InvoiceSequenceCode)
END AS Invoice#,
(ar.OriginalInvoiceAmountOC + CASE
WHEN ROW_NUMBER() OVER (PARTITION BY AccountsReceivableID ORDER BY ar.InvoiceCode) = 1 THEN ISNULL(vat.vatAdjustment, 0)
ELSE 0
END + COALESCE(det.DetailAmountOC, 0)) * COALESCE(cer.CurrencyExchangeRate, ar.CurrencyExchangeRate) AS LocalCurrentAmount,
(ar.OriginalInvoiceAmountOC + CASE
WHEN ROW_NUMBER() OVER (PARTITION BY AccountsReceivableID ORDER BY ar.InvoiceCode) = 1 THEN ISNULL(vat.vatAdjustment, 0)
ELSE 0
END + COALESCE(det.DetailAmountOC, 0)) AS CurrentAmount,
ar.OriginalInvoiceAmountOC + CASE
WHEN ROW_NUMBER() OVER (PARTITION BY AccountsReceivableID ORDER BY ar.InvoiceCode) = 1 THEN ISNULL(vat.vatAdjustment, 0)
ELSE 0
END AS OriginalInvoiceAmountOC,
ar.InvoiceCurrencyCode,
cust.CustomerCode,
UPPER(cust.CustomerName) AS CustomerName
FROM dmbase.fAccountsReceivable ar
INNER JOIN dmbase.dlocation loc ON loc.LocationID = ar.LocationID
INNER JOIN dmbase.dCustomer cust ON cust.CustomerID = ar.CustomerID
LEFT JOIN dmbase.VatAdjustment vat ON ar.InvoiceCode = vat.contractNumber
AND ar.InvoiceSequenceCode = vat.invoiceSequence
AND cust.CustomerCode = vat.CustomerNumber
AND loc.CompanyCode = vat.companyCode
INNER JOIN dmbase.dCompany comp ON (ar.CompanyID = comp.CompanyID)
LEFT JOIN dmbase.dAccountsReceivableInvoiceStatus aris ON (aris.ARInvoiceStatusAMID = ar.ARInvoiceStatusAMID)
LEFT HASH JOIN Details det ON (ar.AccountsReceivableID = det.AccountsReceivableHeaderID)
LEFT JOIN dmbase.dCurrencyExchangeRate cer ON (comp.CompanyCode = cer.CompanyCode
AND ar.InvoiceCurrencyCode = cer.CurrencyCode
AND CASE ar.InvoiceDate WHEN '1900-01-01' THEN GETDATE()ELSE ar.InvoiceDate END BETWEEN cer.ValidFrom AND cer.ValidTo)
LEFT JOIN dmbase.fContractClosedHeader ccd ON ccd.ContractNumber = ar.InvoiceCode
AND ccd.ContractSeqNumber = ar.InvoiceSequenceCode
AND ccd.CompanyID = ar.CompanyID
AND ccd.ContractNumber != '0'
AND ccd.CreditMemoContractNumber != '0'
LEFT JOIN dmbase.fAccountsReceivableHeader cm ON ccd.CreditMemoContractNumber = cm.ContractNumber
AND ccd.CreditMemoSequenceCode = cm.ContractSeqNumber
AND cm.CompanyID = ccd.CompanyID
WHERE (aris.ARInvoiceStatusCode = 'OP'
OR (ar.LastPaymentDate >= GETDATE())
OR (ar.TotalAdjustmentsAmountOC <> 0
AND ar.CurrentAmountLC = 0
AND (ar.OriginalInvoiceAmountOC + ISNULL(vat.vatAdjustment, 0)) + COALESCE(det.DetailAmountOC, 0) <> 0)) --Why ISNULL for one, COALESCE for the other? Both will make the query non-SARGable
AND ar.OriginalInvoiceAmountOC <= 0
AND ar.IsCreditMemo = 'Y' -- ainult Memo (Credit/Debit)
AND cust.InternalCustomerType = 'External'
AND cust.CustomerName NOT IN ('RR AJM', 'RAMIRENT', 'Ramirent')
AND cust.CustomerName NOT LIKE '%[7][0-9][0-9][0-9]%' --A leading wildcard is going to perform slow
AND ar.InvoiceDate <= EOMONTH(DATEADD(DAY, -3, GETDATE())) --I have changed this from EOMONTH(GETDATE(), -3 (which made no sense)
AND NOT EXISTS (SELECT 1
FROM #exceptions1 E
WHERE E.CustomerCode = cust.CustomerCode) --Changed to EXISTS binned UNION you should use UNION ALL if you're doing something like this, it'll be quicker)
AND NOT EXISTS (SELECT 1
FROM #exceptions2 E
WHERE E.CustomerCode = cust.CustomerCode) --Changed to EXISTS binned UNION (you should use UNION ALL if you're doing something like this, it'll be quicker)
AND NOT EXISTS (SELECT 1
FROM #exceptions3 E
WHERE E.CustomerCode = cust.CustomerCode)) --Changed to EXISTS binned UNION you should use UNION ALL if you're doing something like this, it'll be quicker)
SELECT *
FROM Summary
WHERE LocalCurrentAmount NOT BETWEEN -1 AND 1
ORDER BY Invoice_date;
It's worth noting that the above sql is completely untested. I do not have access to your server, or data, so I'm purely relying on "my eye" to note any errors.

Oracle: why is my MW invalidated to NEEDS_COMPILE state after data change in underlying views

We are using following materialized view and underlying views to realize some reporting. The materialized view is refreshed completely manually from our application (by DBMS_SNAPSHOT.REFRESH( '"OVERALL_WEEKLY"','C');). After this call the view is in FRESH state, but after any DML operation is done to underlying tables the materialized view gets NEEDS_COMPILE state.
The queries for views are as follows.
The undetlying tables EBR_CYCLE_TIME and EBR_AREA are changing very frequently. So refresh on commit is not an option for our purpose.
Is there any way to avoid the NEEDS_COMPILE state? Or better what causes the NEED_COMPILE state?
CREATE OR REPLACE FORCE VIEW "OTD_WEEKLY" AS
SELECT
otd.WEEK,
otd.SITE_ID,
otd.AREA_ID,
otd.OTD_METRIC AS CT_METRIC,
ROUND(100 * (SUM(otd.SUCCESS) / SUM(otd.CT_TOTAL_COUNT)), 2) AS OTD_VALUE
FROM
(SELECT
FC.MFL_FISCAL_YR_NUM * 100 + FC.MFL_FISCAL_WK_NUM AS WEEK,
r.BUSINESS_UNIT_ID AS PAL2_ID,
a.site_id AS SITE_ID,
a.area_def_id AS AREA_ID,
a.PRIORITY AS PRIORITY,
r.EBR_BUILD_SUBTYPE AS NPI,
r.CORPORATE_TD AS CTD,
ctd.NAME AS OTD_METRIC,
COUNT(r.ebr_number) AS CT_TOTAL_COUNT,
COUNT(
CASE
WHEN (ct.TIME_ELAPSED) > (ct.TARGET * 86400)
THEN NULL
ELSE r.ebr_number
END) AS SUCCESS,
COUNT(
CASE
WHEN (ct.TIME_ELAPSED) > (ct.TARGET * 86400)
THEN r.ebr_number
ELSE NULL
END) AS MISSED,
COUNT(DISTINCT r.ebr_number) AS TOTAL_NUMBER_OF_EBR
FROM ebr_cycle_time ct
JOIN ebr_area a
ON (a.id = ct.ebr_area_id
AND ct.status = 'FINISHED'
AND a.ship_date IS NOT NULL)
JOIN ebr_request r
ON (a.ebr_id = r.id AND r.submit_date >= to_date((select STRING_VALUE from EBR_STATUS_TABLE where key = 'REPORT_DATE_FROM'),'DD.MM.YY'))
JOIN EBR_GROUP_CYCLE_TIME_DEF gctd
ON (ct.CYCLE_TIME_GROUP_DEF = gctd.ID)
JOIN EBR_CYCLE_TIME_DEF ctd
ON (ctd.ID = gctd.CYCLE_TIME_DEF_ID
AND ctd.OTD_METRIC = 'Y')
JOIN EBR_CYCLE_TIME_GROUP ctg
ON (ctg.id = gctd.CYCLE_TIME_GROUP_ID)
JOIN EEBR_MC_LCL_FISCAL FC
ON (FC.MFL_QUERY_DT = TRUNC(a.ship_date) AND fc.MFL_QUERY_DT > add_months(trunc(sysdate, 'MM'), -8) AND fc.MFL_QUERY_DT <= sysdate)
GROUP BY
r.BUSINESS_UNIT_ID,
a.site_id,
a.area_def_id,
ctd.name,
fc.mfl_fiscal_yr_num,
fc.mfl_fiscal_wk_num,
ct.target,
a.PRIORITY,
r.CORPORATE_TD,
r.EBR_BUILD_SUBTYPE
) otd
GROUP BY
otd.WEEK,
otd.PAL2_ID,
otd.SITE_ID,
otd.AREA_ID,
otd.OTD_METRIC;
CREATE OR REPLACE FORCE VIEW "LAGGING_SCORE_WEEKLY" AS
SELECT
dsc.SITE_ID,
dsc.AREA_ID,
dsc.CT_METRIC,
(dsc.ARCHIVE_YEAR * 100 + dsc.ARCHIVE_WEEK) AS WEEK,
ROUND(AVG(SCORE), 4) AS SCORE
FROM
(SELECT
cts.site_id AS SITE_ID,
ls.AREA_DEF_ID AS AREA_ID,
ctd.name AS CT_METRIC,
ctd.id AS CT_ID,
fc.MFL_QUERY_DT AS ARCHIVE_DAY,
fc.MFL_FISCAL_WK_NUM AS ARCHIVE_WEEK,
fc.MFL_FISCAL_MTH_NUM AS ARCHIVE_MONTH,
fc.MFL_FISCAL_YR_NUM AS ARCHIVE_YEAR,
(fc.MFL_FISCAL_YR_NUM * 100 + fc.MFL_FISCAL_WK_NUM) AS WEEK,
CASE
WHEN SUM(cts.PENALTY) > 0
THEN SUM(cts.PENALTY)
ELSE 0
END AS EBR_PENALTY,
COUNT(DISTINCT cts.ebr_number) AS NUMBER_OF_EBR,
COUNT(DISTINCT (
CASE
WHEN cts.LAGGING_TIME > 0
THEN cts.ebr_number
ELSE NULL
END)) AS NUMBER_OF_LAGGING_EBR,
CASE
WHEN SUM(cts.PENALTY) > 0
THEN greatest(100 - 100 * (SUM(cts.PENALTY) / COUNT(DISTINCT cts.ebr_number)), 0)
ELSE 100
END AS SCORE
FROM EBR_CYCLE_TIME_SNAPSHOT cts
JOIN EBR_REQUEST r
ON (r.ebr_number = cts.ebr_number AND r.submit_date >= to_date((select STRING_VALUE from EBR_STATUS_TABLE where key = 'REPORT_DATE_FROM'),'DD.MM.YY'))
RIGHT JOIN EBR_LAGGING_STATISTIC ls
ON ((TRUNC(ls.stat_date) = TRUNC(cts.SNAPSHOT_TIME))
AND ls.site_id = cts.site_id
AND cts.AREA_DEF_ID = ls.AREA_DEF_ID
AND ls.CT_DEF_ID = cts.CYCLE_TIME_DEF_ID)
JOIN EBR_CYCLE_TIME_DEF ctd
ON (ls.CT_DEF_ID = ctd.id
AND ctd.OTD_METRIC = 'Y')
JOIN EEBR_MC_LCL_FISCAL fc
ON (TRUNC(ls.STAT_DATE) = TRUNC(fc.MFL_QUERY_DT) AND fc.MFL_QUERY_DT > add_months(trunc(sysdate, 'MM'), -8))
GROUP BY
cts.site_id,
ls.AREA_DEF_ID,
ctd.name,
ctd.id,
fc.MFL_QUERY_DT,
fc.MFL_FISCAL_WK_NUM,
fc.MFL_FISCAL_MTH_NUM,
fc.MFL_FISCAL_YR_NUM,
ls.NUMBER_OF_EBR,
ls.NUMBER_OF_LAGGING_EBR,
TRUNC(ls.STAT_DATE)
) dsc
GROUP BY dsc.SITE_ID,
dsc.AREA_ID,
dsc.CT_METRIC,
dsc.ARCHIVE_WEEK,
dsc.ARCHIVE_MONTH,
dsc.ARCHIVE_YEAR;
CREATE OR REPLACE FORCE VIEW "START_COMPLIANCE_WEEKLY" AS
SELECT
'Starts Compliance' AS CT_METRIC,
a.site_id AS SITE_ID,
a.area_def_id AS AREA_ID,
((lstw.MFL_FISCAL_YR_NUM * 100) + lstw.MFL_FISCAL_WK_NUM) AS WEEK,
ROUND(AVG(
CASE
WHEN ((ct.START_DATE IS NOT NULL
AND TRUNC((ct.START_DATE AT TIME ZONE 'MST') AT TIME ZONE s.time_zone) >
TRUNC((a.FIRST_FIRM_START_DATE AT TIME ZONE 'MST') AT TIME ZONE s.time_zone))
OR (ct.START_DATE IS NULL
AND next_day(TRUNC(sysdate) - 7, 'Sun') >
TRUNC((a.FIRST_FIRM_START_DATE AT TIME ZONE 'MST') AT TIME ZONE s.time_zone)))
THEN 0
ELSE 100
END), 2) AS SCORE
FROM ebr_area a
JOIN ebr_request r
ON (a.ebr_id = r.id AND a.FIRST_FIRM_START_DATE IS NOT NULL
AND a.FIRST_FIRM_START_DATE <= next_day(TRUNC(sysdate) - 7, 'Sun') AND AND r.status <> 'CANCELLED'
AND r.submit_date >= to_date((select STRING_VALUE from EBR_STATUS_TABLE where key = 'REPORT_DATE_FROM'),'DD.MM.YY'))
JOIN ebr_site s
ON (s.id = a.site_id)
LEFT JOIN
(SELECT
ct.START_DATE AS START_DATE,
ct.ROUND AS ROUND,
ct.ebr_area_id AS area_id
FROM ebr_cycle_time ct
JOIN EBR_GROUP_CYCLE_TIME_DEF gctd
ON (ct.CYCLE_TIME_GROUP_DEF = gctd.ID
AND ct.status <> 'NEW')
JOIN EBR_CYCLE_TIME_DEF ctd
ON (ctd.ID = gctd.CYCLE_TIME_DEF_ID
AND ctd.code = 'SITE_PROCESSING')
) ct ON (ct.area_id = a.id)
JOIN EEBR_MC_LCL_FISCAL lstw
ON (lstw.MFL_QUERY_DT = TRUNC(FIRST_FIRM_START_DATE) AND lstw.MFL_QUERY_DT > add_months(trunc(sysdate, 'MM'), -8))
GROUP BY
a.site_id,
s.time_zone,
a.area_def_id,
lstw.MFL_FISCAL_YR_NUM,
lstw.MFL_FISCAL_WK_NUM;
CREATE MATERIALIZED VIEW "OVERALL_WEEKLY"
AS SELECT s.code AS SITE_CODE,
s.name AS SITE_NAME,
reports.SITE_ID,
ad.NAME AS AREA,
ad.CODE AS AREA_CODE,
reports.AREA_ID,
reports.REPORT_TYPE,
reports.CT_METRIC,
reports.WEEK,
reports.SCORE
FROM (
(SELECT 'Starts Compliance' AS REPORT_TYPE,
AREA_ID,
SITE_ID,
CT_METRIC,
WEEK,
SCORE
FROM START_COMPLIANCE_WEEKLY
)
UNION
(SELECT 'OTD' AS REPORT_TYPE,
AREA_ID,
SITE_ID,
OTD_WEEKLY.CT_METRIC,
OTD_WEEKLY.WEEK,
OTD_WEEKLY.OTD_VALUE AS SCORE
FROM OTD_WEEKLY
)
UNION
(SELECT 'Lagging' AS REPORT_TYPE,
AREA_ID,
SITE_ID,
LAGGING_SCORE_WEEKLY.CT_METRIC,
LAGGING_SCORE_WEEKLY.WEEK,
LAGGING_SCORE_WEEKLY.SCORE
FROM LAGGING_SCORE_WEEKLY
)) reports
JOIN EBR_SITE s
ON (s.id = reports.SITE_ID)
JOIN EBR_AREA_DEF ad
ON (ad.id = reports.area_id);
This is expected behaviour. The NEEDS_COMPILE status means that the MView data is different than the source data. It's only there for information purposes. If refresh on commit is not suitable for your usage, then you can ignore MViews in NEEDS_COMPILE status.
See MOS Doc ID 264036.1:
Dependencies related to MVs are automatically maintained to ensure
correct operation. When an MV is created, the materialized view
depends on the master tables referenced in its definition. Any DML
operation, such as an INSERT, or DELETE, UPDATE, or DDL operation on
any dependency in the materialized view will cause it to become
invalid.

trying to create view in sql server

Can somebody help me create a view for the following code?
It's showing error near order by
USE [ERPV9]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE VIEW [dbo].[vwSupplierReport]
AS
SELECT
CT.TransDate,
CASE WHEN DN.DocNo IS NOT NULL THEN DN.DocNo ELSE CT.SupplierReference END AS DocNo,
TT.TransTypeName AS [Description],
RP.ChequeNo,
CASE WHEN CT.TransTypeID = 20 THEN (MAX(CT.TotalAmount + CT.TotalTax + CT.Freight)) ELSE 0 END AS Debit,
CASE WHEN CT.TransTypeID IN (21,22) THEN MAX(CT.TotalAmount + CT.TotalTax + CT.Freight) ELSE 0 END AS Credit,
(SUM(MAX(CT.TotalAmount + CT.TotalTax + CT.Freight)) OVER (ORDER BY CT.TransDate, CT.SuppTransID)) AS Balance
FROM SupplierTransactions CT
LEFT JOIN DocumentNos DN ON CT.CompanyID = DN.CompanyID AND CT.TransTypeID = DN.TransTypeID AND CT.TransNo = DN.TransNo
INNER JOIN TransactionTypes TT ON CT.CompanyID = TT.CompanyID AND CT.TransTypeID = TT.TransTypeID
LEFT JOIN ReceiptsAndPayments RP ON CT.CompanyID = RP.CompanyID AND CT.TransTypeID = RP.TransTypeID AND CT.TransNo = RP.TransNo
WHERE CT.CompanyID = 'ccaa3e3f-4486-4465-b5a1-723f647ead17' AND SupplierCode = 'SR0001 ' AND TransDate >= '2013-3-26' AND TransDate < '2015-2-27'
Okay, so you cannot have MAX functions in your CASE statements. MAX is an aggregate and needs a group by clause. Also OVER(ORDER BY columns) is not supported by SQL Server 2008.
USE [ERPV9]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE VIEW [dbo].[vwSupplierReport]
AS
SELECT
CT.TransDate,
CASE WHEN DN.DocNo IS NOT NULL
THEN DN.DocNo
ELSE CT.SupplierReference
END AS DocNo,
TT.TransTypeName AS [Description],
RP.ChequeNo,
CASE WHEN CT.TransTypeID = 20
THEN CT.TotalAmount + CT.TotalTax + CT.Freight
ELSE 0
END AS Debit,
CASE WHEN CT.TransTypeID IN (21,22)
THEN CT.TotalAmount + CT.TotalTax + CT.Freight
ELSE 0
END AS Credit
--SUM(CT.TotalAmount + CT.TotalTax + CT.Freight)
-- OVER (ORDER BY CT.TransDate, CT.SuppTransID) AS Balance
FROM SupplierTransactions CT
LEFT JOIN DocumentNos DN
ON CT.CompanyID = DN.CompanyID
AND CT.TransTypeID = DN.TransTypeID
AND CT.TransNo = DN.TransNo
INNER JOIN TransactionTypes TT
ON CT.CompanyID = TT.CompanyID
AND CT.TransTypeID = TT.TransTypeID
LEFT JOIN ReceiptsAndPayments RP
ON CT.CompanyID = RP.CompanyID
AND CT.TransTypeID = RP.TransTypeID
AND CT.TransNo = RP.TransNo
WHERE CT.CompanyID = 'ccaa3e3f-4486-4465-b5a1-723f647ead17'
AND SupplierCode = 'SR0001 '
AND TransDate >= '2013-3-26'
AND TransDate < '2015-2-27'

joining on count and rank the result t sql

Here's my Count_query:
Declare #yes_count decimal;
Declare #no_count decimal;
set #yes_count=(Select count(*) from Master_Data where Received_Data='Yes');
set #no_count=(Select count(*) from Master_Data where Received_Data='No');
select #yes_count As Yes_Count,#no_count as No_Count,(#yes_count/(#yes_count+#no_count)) As Submission_Count
I am having trouble making joins on these two queries
This is the rest of the query:
Select Distinct D.Member_Id,d.Name,d.Region_Name, D.Domain,e.Goal_Abbreviation,
e.Received_Data, case when Received_Data = 'Service Not Provided' then null
when Received_Data = 'No' then null else e.Improvement end as
Percent_Improvement , case when Received_Data = 'Service Not Provided' then null
when Received_Data = 'No' then null else e.Met_40_20 end as Met_40_20
FROM (
select distinct member_Domains.*,
(case when NoData.Member_Id is null then 'Participating' else ' ' end) as Participating
from
(
select distinct members.Member_Id, members.Name, Members.Region_Name,
case when Domains.Goal_Abbreviation = 'EED Reduction' then 'EED'
When Domains.Goal_Abbreviation = 'Pressure Ulcers' then 'PRU'
when Domains.Goal_Abbreviation = 'Readmissions' then 'READ' else Domains.Goal_Abbreviation end as Domain from
(select g.* from Program_Structure as ps inner join Goal as g on ps.Goal_Id = g.Goal_Id
and ps.Parent_Goal_ID = 0) as Domains
cross join
(select distinct hc.Member_ID, hc.Name,hc.Region_Name from zsheet as z
inner join Hospital_Customers$ as hc on z.CCN = hc.Mcare_Id) as Members
) as member_Domains
left outer join Z_Values_Hospitals as NoData on member_Domains.member_ID = NoData.Member_Id
and Member_Domains.Domain = noData.ReportName) D
Left Outer JOIN
(SELECT B.Member_ID, B.Goal_Abbreviation, B.minRate, C.maxRate, B.BLine, C.Curr_Quarter, B.S_Domain,
(CASE WHEN B.Member_ID IN
(SELECT member_id
FROM Null_Report
WHERE ReportName = B.S_Domain) THEN 'Service Not Provided' WHEN Curr_Quarter = 240 THEN 'Yes' ELSE 'No' END) AS Received_Data,
ROUND((CASE WHEN minRate = 0 AND maxRate = 0 THEN 0 WHEN minRate = 0 AND maxRate > 0 THEN 1 ELSE (((maxRate - minRate) / minRate) * 100) END), .2) AS Improvement,
(CASE WHEN ((CASE WHEN minRate = 0 AND maxRate = 0 THEN 0 WHEN minRate = 0 AND maxRate > 0 THEN 1 ELSE (maxRate - minRate) / minRate END)) <= - 0.4 OR
maxRate = 0 THEN 'Yes' WHEN ((CASE WHEN minRate = 0 AND maxRate = 0 THEN 0 WHEN minRate = 0 AND maxRate > 0 THEN 1 ELSE (maxRate - minRate) / minRate END))
<= - 0.2 OR maxRate = 0 THEN 'Yes' ELSE 'No' END) AS Met_40_20
FROM (SELECT tab.Member_ID, tab.Measure_Value AS minRate, tab.Goal_Abbreviation, A.BLine, tab.S_Domain
FROM Measure_Table_Description AS tab INNER JOIN
(SELECT DISTINCT
Member_ID AS new_memid, Goal_Abbreviation AS new_measure, MIN(Reporting_Period_ID) AS BLine, MAX(Reporting_Period_ID)
AS Curr_Quarter
FROM Measure_Table_Description
WHERE (Member_ID > 1) AND (Measure_Value IS NOT NULL) AND (Measure_ID LIKE '%O%')
GROUP BY Goal_Abbreviation, Member_ID) AS A ON tab.Member_ID = A.new_memid AND tab.Reporting_Period_ID = A.BLine AND
tab.Goal_Abbreviation = A.new_measure) AS B FULL OUTER JOIN
(SELECT tab.Member_ID, tab.Measure_Value AS maxRate, tab.Goal_Abbreviation, A_1.Curr_Quarter
FROM Measure_Table_Description AS tab INNER JOIN
(SELECT DISTINCT
Member_ID AS new_memid, Goal_Abbreviation AS new_measure,
MIN(Reporting_Period_ID) AS BLine, MAX(Reporting_Period_ID)
AS Curr_Quarter
FROM Measure_Table_Description AS Measure_Table_Description_1
WHERE (Member_ID >1) AND (Measure_Value IS NOT NULL) AND (Measure_ID LIKE '%O%')
GROUP BY Goal_Abbreviation, Member_ID) AS A_1 ON tab.Member_ID = A_1.new_memid
AND tab.Reporting_Period_ID = A_1.Curr_Quarter AND
tab.Goal_Abbreviation = A_1.new_measure) AS C ON B.Member_ID = C.Member_ID
WHERE (B.Goal_Abbreviation = C.Goal_Abbreviation) ) E ON D.Member_Id = E.Member_ID AND d.Domain = E.S_Domain
ORDER BY D.Domain,D.Member_ID
How do I get a count of the 'yes'/ (count(yes)+count(no)) for each member_ID as column1 and also display the rank of each member_ID against all the member_IDs in the result as column2. I have come up with a query that generates the count for the entire table, but how do I restrict it each Member_ID.
Thanks for your help.
I haven't taken the time to digest your provided query, but if abstracted to the concept of having an aggregate over a range of data repeated on each row, you should look at using windowing functions. There are other methods, such as using a CTE to do your aggregation and then JOINing back to your detailed data. That might work better for more complex calculations, but the window functions are arguably the more elegant option.
DECLARE #MasterData AS TABLE
(
MemberID varchar(50),
MemberAnswer int
);
INSERT INTO #MasterData (MemberID, MemberAnswer) VALUES ('Jim', 1);
INSERT INTO #MasterData (MemberID, MemberAnswer) VALUES ('Jim', 0);
INSERT INTO #MasterData (MemberID, MemberAnswer) VALUES ('Jim', 1);
INSERT INTO #MasterData (MemberID, MemberAnswer) VALUES ('Jim', 1);
INSERT INTO #MasterData (MemberID, MemberAnswer) VALUES ('Jane', 1);
INSERT INTO #MasterData (MemberID, MemberAnswer) VALUES ('Jane', 0);
INSERT INTO #MasterData (MemberID, MemberAnswer) VALUES ('Jane', 1);
-- Method 1, using windowing functions (preferred for performance and syntactical compactness)
SELECT
MemberID,
MemberAnswer,
CONVERT(numeric(19,4),SUM(MemberAnswer) OVER (PARTITION BY MemberID)) / CONVERT(numeric(19,4),COUNT(MemberAnswer) OVER (PARTITION BY MemberID)) AS PercentYes
FROM #MasterData;
-- Method 2, using a CTE
WITH MemberSummary AS
(
SELECT
MemberID,
SUM(MemberAnswer) AS MemberYes,
COUNT(MemberAnswer) AS MemberTotal
FROM #MasterData
GROUP BY MemberID
)
SELECT
md.MemberID,
md.MemberAnswer,
CONVERT(numeric(19,4),MemberYes) / CONVERT(numeric(19,4),MemberTotal) AS PercentYes
FROM #MasterData md
JOIN MemberSummary ms
ON md.MemberID = ms.MemberID;
First thought is: your query is much, much too complicated. I have spent about 10 minutes now trying to make sense of it and haven't gotten anywhere, so it's obviously going to pose a long-term maintenance challenge to those within your organization going forward as well. I would really recommend you try to find some way of simplifying it.
That said, here is a simplified, general example of how to query on a calculated value and rank the results:
CREATE TABLE member (member_id INT PRIMARY KEY);
CREATE TABLE master_data (
transaction_id INT PRIMARY KEY,
member_id INT FOREIGN KEY REFERENCES member(member_id),
received_data BIT
);
-- INSERT data here
; WITH member_data_counts AS (
SELECT
m.member_id,
(SELECT COUNT(*) FROM master_data d WHERE d.member_id = m.member_id AND d.received_data = 1) num_yes,
(SELECT COUNT(*) FROM master_data d WHERE d.member_id = m.member_id AND d.received_data = 0) num_no
FROM member m
), member_data_calc AS (
SELECT
*,
CASE
WHEN (num_yes + num_no) = 0 THEN NULL -- avoid division-by-zero error
ELSE num_yes / (num_yes + num_no)
END pct_yes
FROM member_data_counts
), member_data_rank AS (
SELECT *, RANK() OVER (ORDER BY pct_yes DESC) AS RankValue
FROM member_data_calc
)
SELECT *
FROM member_data_rank
ORDER BY RankValue ASC;

Resources