I have the complex sub-select statement that slows down my query and also blocks other users.
select
(Case when (Select COUNT(*) from tblQuoteDetails QD where QD.QuoteGUID = a.QuoteGUID) > 1 then
(SELECT Round(Sum(dbo.tblQuoteOptions.Premium),2)
FROM dbo.tblQuotes AS Q
INNER JOIN dbo.lstQuoteStatus ON Q.QuoteStatusID = dbo.lstQuoteStatus.QuoteStatusID
INNER JOIN dbo.tblQuoteOptions ON Q.QuoteGUID = dbo.tblQuoteOptions.QuoteGUID
--INNER JOIN dbo.tblQuoteOptionPremiums ON dbo.tblQuoteOptionPremiums.QuoteOptionGuid = dbo.tblQuoteOptions.QuoteOptionGUID
WHERE (Q.ControlNo = a.ControlNo)
AND (Q.OriginalQuoteGUID IS NULL)
AND (dbo.tblQuoteOptions.Premium <> 0)
AND (DATEDIFF(d,ISNULL(null, dbo.GetEffectiveDate(Q.QuoteGUID)), dbo.GetEffectiveDate(Q.QuoteGUID)) <= 0))
Else
(SELECT Round(Avg(dbo.tblQuoteOptions.Premium),2)
FROM dbo.tblQuotes AS Q
INNER JOIN dbo.lstQuoteStatus ON Q.QuoteStatusID = dbo.lstQuoteStatus.QuoteStatusID
INNER JOIN dbo.tblQuoteOptions ON Q.QuoteGUID = dbo.tblQuoteOptions.QuoteGUID
--INNER JOIN dbo.tblQuoteOptionPremiums ON dbo.tblQuoteOptionPremiums.QuoteOptionGuid = dbo.tblQuoteOptions.QuoteOptionGUID
WHERE (Q.ControlNo = a.ControlNo)
AND (Q.OriginalQuoteGUID IS NULL)
AND (dbo.tblQuoteOptions.Premium <> 0)
AND (DATEDIFF(d,ISNULL(null, dbo.GetEffectiveDate(Q.QuoteGUID)), dbo.GetEffectiveDate(Q.QuoteGUID)) <= 0))
--GROUP BY dbo.tblQuoteOptions.QuoteOptionID
End) As QuotedPremium
FROM tblQuotes a
Not sure if I'm reading execution plan correctly but that's what I see:
Any idea what approach should I take here?
Thanks
Looking into the query completely without having access to your environment won't be terribly efficient, but I can safely say that Key Lookups are expensive, and can often be eliminated by ensuring the columns you're getting from a joined table are INCLUDEd in the index being used. Considering the two key lookups we can amount to almost 80% of the query cost, I'd start there.
Also, part of the issue is the use of DATEDIFF inside a WHERE clause.
AND (DATEDIFF(d,ISNULL(null, dbo.GetEffectiveDate(Q.QuoteGUID))dbo.GetEffectiveDate(Q.QuoteGUID)) <= 0))
This will severely hamper the optimizer from doing it's job. Simplifying this particular comparison could make a big difference.
Related
I need to return all vendors regardless of whether there has been a purchase from that vendor. The query is currently only returning records where the vendor had a purchase.
SELECT vendors.NAME,
Iif([fundingsourceid] = 10, [amount], 0) AS Credit,
Iif(( [fundingsourceid] = 2 )
OR ( [fundingsourceid] = 3 ), [amount], 0) AS EBT,
Iif([fundingsourceid] = 4, [amount], 0) AS [Match],
cardpurchases.updateddate
FROM vendors
FULL OUTER JOIN cardpurchases
ON cardpurchases.vendorid = vendors.vendorid
LEFT JOIN cardfundings
ON cardpurchases.cardfundingid = cardfundings.cardfundingid
INNER JOIN marketevents
ON cardpurchases.marketeventid = marketevents.marketeventid
INNER JOIN markets
ON marketevents.marketid = markets.marketid
WHERE (cardpurchases.updateddate >= '10/22/2014' OR cardpurchases.updateddate IS NULL)
AND (cardpurchases.updateddate < '10/23/2014' OR cardpurchases.updateddate IS NULL)
AND (markets.marketid = 47 OR markets.marketid IS NULL)
ORDER BY vendors.NAME
Although you have specified a FULL OUTER JOIN later in your query you are restricting the resultset based on columns in the cardpurchases table which is causing vendors which have no cardpurchases to disappear.
You can do either of the following:
WHERE
((cardpurchases.updateddate >= '10/22/2014'
AND cardpurchases.updateddate < '10/23/2014')
OR cardpurchases.updateddate IS NULL)
AND markets.marketid = 47
Or
FROM vendors
LEFT JOIN cardpurchases
ON cardpurchases.vendorid = vendors.vendorid
AND cardpurchases.updateddate >= '10/22/2014'
AND cardpurchases.updateddate < '10/23/2014')
You need to account for NULLs in your WHERE clause:
WHERE (cardpurchases.updateddate >= '10/22/2014' OR cardpurchases.updateddate IS NULL)
AND (cardpurchases.updateddate < '10/23/2014' OR cardpurchases.updateddate IS NULL)
AND (markets.marketid = 47 OR markets.marketid IS NULL)
You also should use parentheses to control the join so that the later INNER JOINs don't poison it:
FROM vendors
FULL OUTER JOIN (cardpurchases
LEFT JOIN cardfundings
ON cardpurchases.cardfundingid = cardfundings.cardfundingid
INNER JOIN marketevents
ON cardpurchases.marketeventid = marketevents.marketeventid
INNER JOIN markets
ON marketevents.marketid = markets.marketid)
ON cardpurchases.vendorid = vendors.vendorid
That's only partial, since you'll experience the same issue with the LEFT JOIN as you do with the FULL.
The following is my query:
SELECT
U.Part
,U.EndDate
,F.Part
,F.EndDate
,(U.Calc1/(CASE F.Calc2 WHEN 0 THEN .00000001 ELSE F.Calc2 END)) AS PercentRepaired
,(1-(U.Calc1/(CASE F.Calc2 WHEN 0 THEN .00000001 ELSE F.Calc2 END))) AS PercentNoFault
,NF.*
,R.*
FROM
RCCalc1Part U
INNER JOIN
RCCalc2Part F
ON
U.Part = F.Part AND U.EndDate = F.EndDate
INNER JOIN RCNF NF
ON Part = LEFT(NF.Part,7)
INNER JOIN Repair R
ON
NF.Part = (CASE LEFT(NF.Part,7) WHEN '2000000' THEN CONCAT(R.Part,'-',LEFT(R.ExtendedPart,1)) ELSE NF.Part END)
ORDER BY
U.SN ASC, NF.Part ASC, U.EndDate ASC
The following is the INNER JOIN that I am concerned about:
INNER JOIN Repair R
ON
NF.Part = (CASE LEFT(NF.Part,7) WHEN '2000000' THEN CONCAT(R.Part,'-',LEFT(R.ExtendedPart,1)) ELSE NF.Part END)
I am trying to use a debugger to find the value of the right side of the =. It doesn't seem to be doing what I want it to. When NF. Part = 2000000 then I want to CONCAT R.Part and want that concatenation to be the value on the right side of the =.
Am I doing this right? If not...am I far off?
Thanks.
INNER JOIN Repair R ON NF.Part = (CASE LEFT(NF.Part,7) WHEN '2000000' THEN CONCAT(R.Part,'-',LEFT(R.ExtendedPart,1)) ELSE R.Part END) was what I was looking for....everything was right except the column after the else statement.
This is the schema :
And this is the sql that as I understand is too complex for SQL Optimizer:
SELECT * FROM
(
select pp.Id as PaymentPartId, b.Id as BudgetId, grouping(bp.ID) as g1 , sum(pp.Amount) PaymentsSum, sum(bp.Amount) BudgetSum
from Projects pr
inner join Payments p ON pr.Id = p.ProjectID
inner join PaymentParts pp ON p.Id = pp.PaymentId
inner join Budgets b ON pr.Id = b.ProjectID
inner join Budgetparts bp ON b.Id = bp.BudgetId
group by pp.Id, b.Id, rollup(bp.ID)
) x
WHERE x.PaymentPartId = 777
SQLFIDDLE: http://sqlfiddle.com/#!6/aa74e/11 (with autogenerated data)
What I expect: execution plan should contain index seek on x.PaymentPartId. Why? Because this query is equivalent to:
select pp.Id as PaymentPartId, b.Id as BudgetId, grouping(bp.ID) as g1, sum(pp.Amount) PaymentsSum, sum(bp.Amount) BudgetSum
from Projects pr
inner join Payments p ON pr.Id = p.ProjectID
inner join PaymentParts pp ON p.Id = pp.PaymentId
inner join Budgets b ON pr.Id = b.ProjectID
inner join Budgetparts bp ON b.Id = bp.BudgetId
WHERE pp.Id = 777
group by pp.Id, b.Id, rollup(bp.ID)
...and the last query uses index seek.
But SQL Optimizer not only refuse to use the index but ignore all hints (I propose you to expirement wiht sqlfiddle - it is really interesting).
So the question is: am I right that it is impossible to force SQL Server Optimizer to use index seek there? It seems like rollup is something that split sql optimizer "optimization frame" to two parts and it makes it impossible to optimize WHOLE query.
P.S. For those who votes for closing this "non-programming question": try to put optimizer hints (sqlfiddle is ready to test your programming skills!).
Why hints doesn't work? - Roman Pokrovskij
It is in the documentation:
http://technet.microsoft.com/en-us/library/ms181714.aspx
Query hints can be specified only in the top-level query, not in subqueries
I tried everything but I couldn't overcome this problem.
I have a table-valued function.
When I call this function with
SELECT * FROM Ratings o1
CROSS APPLY dbo.FN_RatingSimilarity(50, 497664, 'Cosine') o2
WHERE o1.trackId = 497664
It takes a while to be executed. But when I do this.
SELECT * FROM Ratings o1
CROSS APPLY dbo.FN_RatingSimilarity(50, o1.trackId, 'Cosine') o2
WHERE o1.trackId = 497664
It is executed in 32 seconds. I created all indexes but It didn't help.
My function by the way:
ALTER FUNCTION [dbo].[FN_RatingSimilarity]
(
#trackId INT,
#nTrackId INT,
#measureType VARCHAR(100)
)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN
(
SELECT o2.id,
o2.name,
o2.releaseDate,
o2.numberOfRatings,
o2.averageRating,
COUNT(1) as numberOfSharedUsers,
CASE #measureType
WHEN 'Cosine' THEN SUM(o3.score*o4.score)/(0.01+SQRT(SUM(POWER(o3.score,2))) * SQRT(SUM(POWER(o4.score,2))))
WHEN 'AdjustedCosine' THEN SUM((o3.score-o5.averageRating)*(o4.score-o5.averageRating))/(0.01+SQRT(SUM(POWER(o3.score-o5.averageRating, 2)))*SQRT(SUM(POWER(o4.score-o5.averageRating, 2))))
WHEN 'Pearson' THEN SUM((o3.score-o1.averageRating)*(o4.score-o2.averageRating))/(0.01+SQRT(SUM(POWER(o3.score-o1.averageRating, 2)))*SQRT(SUM(POWER(o4.score-o2.averageRating, 2))))
END as similarityRatio
FROM dbo.Tracks o1
INNER JOIN dbo.Tracks o2 ON o2.id != #trackId
INNER JOIN dbo.Ratings o3 ON o3.trackId = o1.id
INNER JOIN dbo.Ratings o4 ON o4.trackId = o2.id AND o4.userId = o3.userId
INNER JOIN dbo.Users o5 ON o5.id = o4.userId
WHERE o1.id = #trackId
AND o2.id = ISNULL(#nTrackId, o2.id)
GROUP BY o2.id,
o2.name,
o2.releaseDate,
o2.numberOfRatings,
o2.averageRating
)
Any help would be appreciated.
Thanks.
Emrah
I believe that your bottleneck is the calculations + your very expensive inner joins.
The way your are joining is basically creating a cross join - It is returning a result set with all ther records linked to all other records, Except the one for which the id is supplied. Then you go and add to that result set with the other inner joins.
For every inner join, SQL goes and creates a result set with all the rows matching.
So the first thing you do in your query is to tell SQL to basically do a cross join on the same table. (I am assuming you are still following, that looks pretty advanced so I'll just take you are familiar with advanced SQL syntax and operators)
Now in the next inner join, you are applying the Results table to your newly created huge result set, and only then filtering out the ones not both tables.
So as a start, see if you can't do your joins the other way around. (This really depends on your table record count and record sizes). Try to have the smallest result sets first and then join onto that.
The second thing you might want to try is to firstly limit your result set even before the joins.So start with a CTE where you filter for o1.id = #trackId. Then select * from this CTE , do your joins on the CTE and then filter in your query for o2.id = ISNULL(#nTrackId, o2.id)
I will work on an example, stay tuned...
--
Ok, I added an example, did a quick test and the values returned are the same. Run this through your data and let us know if there is any improvement. (Note, this does not address the INNER JOIN order point discussed, still do play around with that.)
Example:
ALTER FUNCTION [dbo].[FN_RatingSimilarity_NEW]
(
#trackId INT,
#nTrackId INT,
#measureType VARCHAR(100)
)
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN
(
WITH CTE_ALL AS
(
SELECT id,
name,
releaseDate,
numberOfRatings,
averageRating
FROM dbo.Tracks
WHERE id = #trackId
)
SELECT o2.id,
o2.name,
o2.releaseDate,
o2.numberOfRatings,
o2.averageRating,
COUNT(1) as numberOfSharedUsers,
CASE #measureType
WHEN 'Cosine' THEN SUM(o3.score*o4.score)/(0.01+SQRT(SUM(POWER(o3.score,2))) * SQRT(SUM(POWER(o4.score,2))))
WHEN 'AdjustedCosine' THEN SUM((o3.score-o5.averageRating)*(o4.score-o5.averageRating))/(0.01+SQRT(SUM(POWER(o3.score-o5.averageRating, 2)))*SQRT(SUM(POWER(o4.score-o5.averageRating, 2))))
WHEN 'Pearson' THEN SUM((o3.score-o1.averageRating)*(o4.score-o2.averageRating))/(0.01+SQRT(SUM(POWER(o3.score-o1.averageRating, 2)))*SQRT(SUM(POWER(o4.score-o2.averageRating, 2))))
END as similarityRatio
FROM CTE_ALL o1
INNER JOIN dbo.Tracks o2 ON o2.id != #trackId
INNER JOIN dbo.Ratings o3 ON o3.trackId = o1.id
INNER JOIN dbo.Ratings o4 ON o4.trackId = o2.id AND o4.userId = o3.userId
INNER JOIN dbo.Users o5 ON o5.id = o4.userId
WHERE o2.id = ISNULL(#nTrackId, o2.id)
GROUP BY o2.id,
o2.name,
o2.releaseDate,
o2.numberOfRatings,
o2.averageRating
)
SELECT
*
FROM
RM_Sales_Union
WHERE
DOCDATE >= 'September 1, 2011'
AND DOCDATE < 'October 1, 2011'
AND CUSTNMBR = '2186020'
That query results in an error:
Msg 8134, Level 16, State 1, Line 1
Divide by zero error encountered.
if run as shown.
Excluding the last line AND CUSTNMBR = '2186020' allows the query to complete.
CUSTNMBR is a char(21) field. Divide by zero has me confused.
What is the "correct" way to work around this?
RM_Sales_Union is a union query view:
SELECT ACTNUMBR_1,
ACTNUMBR_2,
ACTNUMBR_3,
ORSLSAMT,
CUSTCLAS,
CUSTNAME,
CUSTNMBR,
SLPRSNID,
DOCABREV,
CSPORNBR,
CURNCYID,
DOCDATE,
DOCNUMBR,
GLPOSTDT,
SLSAMNT,
VOIDSTTS,
SLPRSNFN,
SPRSNSLN,
DocOrigin,
ORFRTAMT,
FRTAMNT,
COMPRCNT,
TRDISAMT,
ORTDISAM,
ORMISCAMT,
ORTAXAMT,
ORCTRXAM
FROM dbo.RM_Sales_Hist
UNION
SELECT ACTNUMBR_1,
ACTNUMBR_2,
ACTNUMBR_3,
ORSLSAMT,
CUSTCLAS,
CUSTNAME,
CUSTNMBR,
SLPRSNID,
DOCABREV,
CSPORNBR,
CURNCYID,
DOCDATE,
DOCNUMBR,
GLPOSTDT,
SLSAMNT,
VOIDSTTS,
SLPRSNFN,
SPRSNSLN,
DocOrigin,
ORFRTAMT,
FRTAMNT,
COMPRCNT,
TRDISAMT,
ORTDISAM,
ORMISCAMT,
ORTAXAMT,
ORCTRXAM
FROM dbo.RM_Sales_Open
RM_Sales_Hist and RM_Sales_Open are views defined as follows:
--RM_Sales_Hist
SELECT dbo.GL_Sales_Accounts.ACTNUMBR_1,
dbo.GL_Sales_Accounts.ACTNUMBR_2,
dbo.GL_Sales_Accounts.ACTNUMBR_3,
ISNULL(dbo.MC020102.ORSLSAMT, dbo.RM30101.SLSAMNT) AS ORSLSAMT,
dbo.RM00101.CUSTCLAS,
dbo.RM00101.CUSTNAME,
dbo.RM00101.CUSTNMBR,
dbo.RM00101.SLPRSNID,
dbo.RM40401.DOCABREV,
dbo.RM30101.CSPORNBR,
dbo.RM30101.CURNCYID,
dbo.RM30101.DOCDATE,
dbo.RM30101.DOCNUMBR,
dbo.RM30101.GLPOSTDT,
dbo.RM30101.SLSAMNT,
dbo.RM30101.VOIDSTTS,
dbo.RM00301.SLPRSNFN,
dbo.RM00301.SPRSNSLN,
'HIST' AS DocOrigin,
ISNULL(dbo.MC020102.ORFRTAMT, dbo.RM30101.FRTAMNT) AS ORFRTAMT,
dbo.RM30101.FRTAMNT,
dbo.RM00301.COMPRCNT,
dbo.RM30101.TRDISAMT,
ISNULL(dbo.MC020102.ORTDISAM, 0) AS ORTDISAM,
ISNULL(dbo.MC020102.ORMISCAMT, 0) AS ORMISCAMT,
ISNULL(dbo.MC020102.ORTAXAMT, 0) AS ORTAXAMT,
ISNULL(dbo.MC020102.ORCTRXAM, 0) AS ORCTRXAM,
dbo.RM00101.STATE
FROM dbo.GL_Sales_Accounts
INNER JOIN dbo.RM30301
ON dbo.GL_Sales_Accounts.DSTINDX = dbo.RM30301.DSTINDX
INNER JOIN dbo.RM30101
ON dbo.RM30301.CUSTNMBR = dbo.RM30101.CUSTNMBR
AND dbo.RM30301.DOCNUMBR = dbo.RM30101.DOCNUMBR
INNER JOIN dbo.RM00101
ON dbo.RM30101.CUSTNMBR = dbo.RM00101.CUSTNMBR
INNER JOIN dbo.RM40401
ON dbo.RM30101.RMDTYPAL = dbo.RM40401.RMDTYPAL
INNER JOIN dbo.RM00301
ON dbo.RM00101.SLPRSNID = dbo.RM00301.SLPRSNID
LEFT OUTER JOIN dbo.MC020102
ON dbo.RM30301.DOCNUMBR = dbo.MC020102.DOCNUMBR
WHERE ( CAST(dbo.RM30301.DOCNUMBR AS VARCHAR(21)) NOT IN (SELECT
CAST(
DOCNUMBR AS
VARCHAR(21)) AS
Expr1
FROM
dbo.Invoices_With_Display_Discounts) )
--RM_Sales_Open
SELECT dbo.GL_Sales_Accounts.ACTNUMBR_1,
dbo.GL_Sales_Accounts.ACTNUMBR_2,
dbo.GL_Sales_Accounts.ACTNUMBR_3,
ISNULL(dbo.MC020102.ORSLSAMT, 0) AS ORSLSAMT,
dbo.RM00101.CUSTCLAS,
dbo.RM00101.CUSTNAME,
dbo.RM00101.CUSTNMBR,
dbo.RM00101.SLPRSNID,
dbo.RM40401.DOCABREV,
dbo.RM20101.CSPORNBR,
dbo.RM20101.CURNCYID,
dbo.RM20101.DOCDATE,
dbo.RM20101.DOCNUMBR,
dbo.RM20101.GLPOSTDT,
dbo.RM20101.SLSAMNT,
dbo.RM20101.VOIDSTTS,
dbo.RM00301.SLPRSNFN,
dbo.RM00301.SPRSNSLN,
'OPEN' AS DocOrigin,
ISNULL(dbo.MC020102.ORFRTAMT, 0) AS ORFRTAMT,
dbo.RM20101.FRTAMNT,
dbo.RM00301.COMPRCNT,
dbo.RM20101.TRDISAMT,
ISNULL(dbo.MC020102.ORTDISAM, 0) AS ORTDISAM,
ISNULL(dbo.MC020102.ORMISCAMT, 0) AS ORMISCAMT,
ISNULL(dbo.MC020102.ORTAXAMT, 0) AS ORTAXAMT,
ISNULL(dbo.MC020102.ORCTRXAM, 0) AS ORCTRXAM,
dbo.RM00101.STATE
FROM dbo.GL_Sales_Accounts
INNER JOIN dbo.RM10101
ON dbo.GL_Sales_Accounts.DSTINDX = dbo.RM10101.DSTINDX
INNER JOIN dbo.RM20101
ON dbo.RM10101.CUSTNMBR = dbo.RM20101.CUSTNMBR
AND dbo.RM10101.DOCNUMBR = dbo.RM20101.DOCNUMBR
INNER JOIN dbo.RM00101
ON dbo.RM20101.CUSTNMBR = dbo.RM00101.CUSTNMBR
INNER JOIN dbo.RM40401
ON dbo.RM20101.RMDTYPAL = dbo.RM40401.RMDTYPAL
INNER JOIN dbo.RM00301
ON dbo.RM00101.SLPRSNID = dbo.RM00301.SLPRSNID
LEFT OUTER JOIN dbo.MC020102
ON dbo.RM10101.DOCNUMBR = dbo.MC020102.DOCNUMBR
WHERE ( CAST(dbo.RM20101.DOCNUMBR AS VARCHAR(21)) NOT IN (SELECT
CAST(
DOCNUMBR AS
varchar(21)) AS
Expr1
FROM
dbo.Invoices_With_Display_Discounts) )
I guess that RM_Sales_Union is a view that contains a division. The filter simply changes something like a COUNT or SUM to give zero, and it is this that is used as divisor
Change it to use something/NULLIF(..., 0) to make the expressions give NULL. Or ISNULL(something/NULLIF(..., 0), 0) to get NULL instead