I got an app running on my SQL Server that is starting to slow down on a specific task. I ran SQL Profiler and noticed that the
following query is taking an enormous (1-2 minutes) amount of time. I don't have access to the code to change the query.
Is there anything I can tune/change in the database? The PC10000 table in the statement below has approx. 119000 records. I also have the execution plan attached.
SELECT TOP 25
zProjectID, zTaskID, zTransactionNumber, zTransactionDate, zUserID,
zCostCategoryDDL, zCostCategoryString, zSubCostCategory, zSubCostCategoryString,
zDepartmentID, zJournalEntry, zPostingDate, zSalesPostingDate, zPeriodNumber,
zTransactionDescription, zBillingDescriptionLine1, zBillingDescriptionLine2,
zBillingDescriptionLine3, zBillingDescriptionLine4, zSalesAccountIndex,
zSalesAccountString, zDistDocumentTypeDDL, zDistDocumentNumber, zDistSequenceNumber,
zSalesDocumentTypeDDL, zSalesDocumentNumber, zSalesLineNumber, zDistHistoryYear,
zSeriesDDL, zSourceDoc, zWebSource, zOrigDocumentNumber, zOrigDocumentDate,
zOrigID, zOrigName, zExpenseStatusDDL, zApprovalUserIDCost, zAccountIndex,
zAccountNumberString, zBillingStatusDDL, zApprovalUserIDBilling, zBillingWorkQty,
zBillingWorkAmt, zQty, zQtyBilled, zUnitCost,
zUnitPrice, zRevenueAmt, zOriginatingRevenueAmt, zCostAmtEntered, zCostAmt,
zOriginatingCostAmt, zPayGroupID, zPayrollStatusDDL, zTotalTimeStatusDDL,
zEmployeeID, zHoursEntered, zHoursPaid, zPayRecord, zItemID, zItemDescription,
zUofM, zItemQty, zBurdenStatusDDL, zUserDefinedDate, zUserDefinedDate2,
zUserDefinedString, zUserDefinedString2, zUserDefinedCurrency,
zUserDefinedCurrency2, zNoteIndex, zImportType, DEX_ROW_ID
FROM
DBServer.dbo.pc10000
WHERE
(zDistDocumentNumber in
(select cast(JRNENTRY as varchar(20))
from DBServer..GL10001
where BACHNUMB = 'PMCHK00004283')
or zSalesDocumentNumber in
(select cast(JRNENTRY as varchar(20))
from DBServer..GL10001
where BACHNUMB = 'PMCHK00004283'))
ORDER BY
zProjectID ASC ,zTaskID ASC ,zTransactionNumber ASC
The biggest problem you have looks to be due to lack of suitable indexes.
You can see that because of the presence of Table Scans within the execution plan.
Table Scans hit performance as they mean the whole table is being scanned for data that matches the given clauses in the query.
I'd recommend you add an index on BACHNUMB in GL10001
You may also want to try indexes on zDistDocumentNumber and zSalesDocumentNumber in PC10000, but I think the GL10001 index is the main one.
"IN" clauses are typically quite expensive compared to other techniques, but as you can't change the query itself then there's nothing you can do about that.
Without a doubt, you need to add suitable indexes
The query is doing 2 table scans on the GL10001 table. From a quick look at the query (which is a bit hard to read) I would see if you have an index on the BACHNUMB column.
the execution plan shows pretty clearly that actually locating the rows is what's taking all the time (no cumbersome bookmark lookups, or aggregation/rearrange tasks), so it's quite positively going to be a question of indexing. hover the table scans in the execution plan, and check 'object' in the tooltip, to see what columns are being used. see to it that they're indexed.
you might also want to run a trace to sample some live data, and feed that to the database tuning advisor.
You could rewrite those sub-selects as a join, and add an index to GP01..GL10001 on BACHNUMB and JRNENTRY
Since you can't change the query, the best thing you could do is make sure you have indexes on the columns that you're using for your joins (and subqueries). If you can think of a better query plan, you could provide that to SQL Server instead of letting it calculate its own (this is a very rare case).
Replace the OR with a UNION ALL of two queries this should get shot of those spools
i.e. run the query once with something like this
SELECT ....
(zDistDocumentNumber in
(select cast(JRNENTRY as varchar(20))
from DBServer..GL10001
where BACHNUMB = 'PMCHK00004283')
UNION ALL
SELECT ...
zSalesDocumentNumber in
(select cast(JRNENTRY as varchar(20))
from DBServer..GL10001
where BACHNUMB = 'PMCHK00004283'))
In addition to adding indexes, you can also convert the IN statements to EXISTS... something along these lines:
SELECT TOP 25 ....
FROM GP01.dbo.pc10000 parent
WHERE EXISTS
(
SELECT child.*
FROM GP01..GL10001 child
WHERE BACHNUMB = 'PMCHK00004283'
and parent.zDistDocumentNumber = child.JRNENTRY
)
OR EXISTS
(
SELECT child2.*
FROM GP01..GL10001 child2
WHERE BACHNUMB = 'PMCHK00004283'
and parent.zSalesDocumentnumber = child2.JRENTRY
)
ORDER BY zProjectID ASC ,zTaskID ASC ,zTransactionNumber ASC
Related
I'm having a huge issue on a SQL query, after I added an index.
declare #DateFromCT date, #DateToCT date;
declare #DateFromCT2 date, #DateToCT2 date;
set dateformat dmy;
set #DateFromCT= '1/1/2015'; set #DateToCT= '31/3/2015';
set #DateFromCT2= '1/4/2015'; set #DateToCT2= '30/4/2015';
Select distinct CT.CodCliente,ct.codacesso FROM CT_Contabilidade CT
Inner join CD_PlanoContas PC ON CT.CodAcesso = PC.Cod
WHERE NOT exists (
SELECT 1 FROM ct_contabilidade CT2
WHERE CT2.CodAcesso = CT.CodAcesso
and CT2.Data between #DateFromCT2 and #DateToCT2
And ( CT2.CodEmpresa = 1) And CT2.codcliente = ct.codcliente )
and CT.Data between #DateFromCT and #DateToCT
AND PC.subgrupo = 'C'
And ( CT.CodEmpresa = 1 ) And ct.codCliente > 0
The CT_Contabilidade's PK is a Sequential (bigint identity), clustered index.
It has 1.5 million records.
Without other non-clustered indexes, it performs well, took less than 1 second. That's OK for me.
I create an index over the CodAcesso to match CD_PlanoContas key (cod);
The CD_PlanoContas PK (clustered index) is Cod.
It's still performing well. No notable difference...
So I create a index over the codCliente (since it also refers another table)
... And after this, the query is TOO slow; it is taking 7 or 8 MINUTES.
If I drop the CodAcesso index, it turn to be ok.
If I drop the CodCliente index, it is ok too.
If I let them both, but change the query , taking of the Inner Join with CD_Planocontas (and consequently , the filter "AND PC.subgrupo = 'C'") it is OK.
I can't imagine the indexes are causing the query to behave that way.
It's a HUGE difference, not just a "loss of performance". I try some other things, as take out each filter... not changed.
The execution plan suggests an index:
CREATE NONCLUSTERED INDEX [<Name of Missing Index, sysname,>]
ON [dbo].[CT_Contabilidade] ([CodEmpresa],[Data],[CodCliente])
INCLUDE ([CodAcesso])
I created it, and the query works fine, even with the 2 other indexes (codCliente and codAcesso)
But I didn't like to create a specific index to this query (it's just one of many queries that uses these tables).
If runs well without no index, I think it should runs at least EQUAL with this 2 indexes.
What causes the performance to change so drastically? What do I need to change to speed things up?
try using an index optimizer hint to control which index is being used.
for example:
select *
from titles with (index (titleind))
where title = 'The Gourmet Microwave'
use the 'set statistics io on' command to see the number of pages being scanned with each query/index combo and use the 'rightclick/show execution plan' option to see how the query is being executed
It is not always good idea to follow the suggestion of execution plan.
I suggest you compare the execution plan before and after adding index and see the difference. Maybe that index cause SQL engine to choose a bad plan.
Also try update statistics on your table and index and see how if affects.
I have the below sql
SELECT Cast(Format(Sum(COALESCE(InstalledSubtotal, 0)), 'F') AS MONEY) AS TotalSoldNet,
BP.BoundProjectId AS ProjectId
FROM BoundProducts BP
WHERE ( BP.IsDeleted IS NULL
OR BP.IsDeleted = 0 )
GROUP BY BP.BoundProjectId
I already have an index on the table BoundProducts on this column order (BoundProjectId, IsDeleted)
Currently this query takes around 2-3 seconds to return the result. I am trying to reduce it to zero seconds.
This query returns 25077 rows as of now.
Please provide me any ideas to improvise the query.
Looking at this in a bit different point of view, I can think that your OR condition is screwing up your query, why not to rewrite it like this?
SELECT CAST(FORMAT(SUM(COALESCE(BP.InstalledSubtotal, 0)), 'F') AS MONEY) AS TotalSoldNet
, BP.BoundProjectId AS ProjectId
FROM (
SELECT BP.BoundProjectId, BP.InstalledSubtotal
FROM dbo.BoundProducts AS BP
WHERE BP.IsDeleted IS NULL
UNION ALL
SELECT BP.BoundProjectId, BP.InstalledSubtotal
FROM dbo.BoundProducts AS BP
WHERE BP.IsDeleted = 0
) AS BP
GROUP BY BP.BoundProjectId;
I've had better experience with UNION ALL rather than OR.
I think it should work totally the same. On top of that, I'd create this index:
CREATE NONCLUSTERED INDEX idx_BoundProducts_IsDeleted_BoundProjectId_iInstalledSubTotal
ON dbo.BoundProducts (IsDeleted, BoundProjectId)
INCLUDE (InstalledSubTotal);
It should satisfy your query conditions and seek index quite well. I know it's not a good idea to index bit fields, but it's worth trying.
P.S. Why not to default your IsDeleted column value to 0 and make it NOT NULLABLE? By doing that, it should be enough to do a simple check WHERE IsDeleted = 0, that'd boost your query too.
If you really want to try index seek, it should be possible using query hint forceseek, but I don't think it's going to make it any faster.
The options I suggested last time are still valid, remove format and / or create an indexed view.
You should also test if the problem is the query itself or just displaying the results after that, for example trying it with "select ... into #tmp". If that's fast, then the problem is not the query.
The index name in the screenshot is not the same as in create table statement, but I assume that's just a name you changed for the question. If the scan is happening to another index, then you should include that too.
-- Holds last 30 valdates
create table #valdates(
date int
)
insert into #valdates
select distinct top (30) valuation_date
from tbsm.tbl_key_rates_summary
where valuation_date <= 20150529
order by valuation_date desc
select
sum(fv_change), sc_group, valuation_date
from
(select *
from tbsm.tbl_security_scorecards_summary
where valuation_date in (select date from #valdates)) as fact
join
(select *
from tbsm.tbl_security_classification
where sc_book = 'UC' ) as dim on fact.classification_id = dim.classification_id
group by
valuation_date, sc_group
drop table #valdates
This query takes around 40 seconds to return because the fact table has almost 13 million rows.. Can I do anything about this?
Based on the fact that there's no proper index that supports the fetch, that's probably the easiest (or only) option to really improve the performance. Most likely index like this would improve the situation a lot:
create index idx_security_scorecards_summary_1 on
tbl_security_scorecards_summary (valuation_date, classification_id)
include (fv_change)
Everything depends of course on how good the selectivity of the valuation_date and classification_id fields are (=how big portion of the table needs to be fetched) and might work better with the fields in opposite order. The field fv_change is in the include section so that it's included in the index structure so there's no need to fetch it from the base table.
Include fields help if the SQL has to fetch a lot of rows from the table. If the amount of rows that this touches is small, then it might not help at all. Like always in indexing, this of course slows down the inserts / updates, and is optimized for this case only and you should of course look at the bigger picture too.
The select is written in a little bit strange way, not sure if that makes any difference, but you could also try the normal way to do this:
select
sum(fact.c), dim.sc_group, fact.valuation_date
from
tbsm.tbl_security_scorecards_summary fact
join tbsm.tbl_security_classification dim
on fact.classification_id = dim.classification_id
where
fact.valuation_date in (select date from #valdates) and
dim.sc_book = 'UC'
group by
fact.valuation_date,
dim.sc_group
Looking at "statistics io" output should give you a good idea which table is causing the slowness, and looking at query plan to see if there's any strange operators might help to understand the situation better.
I'm seeing how we can improve the performance of the following sybase query. Currently it takes about 1.5 hrs.
CREATE TABLE #TempTable
(
T_ID numeric,
M_ID numeric,
M_USR_NAME char(10),
M_USR_GROUP char(10),
M_CMP_DATE datetime,
M_CMP_TIME numeric,
M_TYPE char(10),
M_ACTION char(15),
)
select
T.M_USR_NAME,
T.M_USR_GROUP,
T.M_CMP_DATE,
T.M_CMP_TIME,
T.M_TYPE,
T.M_ACTION
from #TempTable T, AUD_TN B
where T.M_ID=B.M_ID
and T.T_ID in
(
select M_NB from TRN H where (M_BENTITY ="KROP" or M_SENTITY = "KROP")
)
UNION
select
A.M_USR_NAME,
A.M_USR_GROUP,
A.M_DATE_CMP,
A.M_TIME_CMP,
A.M_TYPE,
A.M_ACTION
from AUD_VAL A, TRN H
where A.M_DATE_CMP >= '1 May 2012' and A.M_DATE_CMP <= '31 May 2012'
and A.M_ACT_NB0=H.M_NB
and (H.M_BENTITY ="KROP" or H.M_SENTITY = "KROP")
UNION
select
TR.M_USR_NAME,
TR.M_USR_GROUP,
TR.M_DATE_CMP,
TR.M_TIME_CMP,
TR.M_TYPE,
TR.M_ACTION
from TRN_AUD TR, TRN H
where TR.M_DATE_CMP >= '1 May 2012' and TR.M_DATE_CMP <= '31 May 2012'
and TR.M_ACT_NB0=H.M_NB
and (H.M_BENTITY ="KROP" or H.M_SENTITY = "KROP")
DROP table #TempTable
Any help is greatly appreciated. Please note the following
The only table which is not indexed above is AUD_TN
Cheers
RC
Presumably the temporary table is populated, and with a lot of rows?
The temp doesn't need to be indexed but all joins in that part will need to use indexes.
Why not try each part of the UNION separately to find if one of them's slow?
Are you okay using SET SHOWPLAN ON? I think you need to be able to do that as well probably - you need to be able to check that Sybase is using indexes to join right.
TRN BENTITY and SENTITY - indexed? If not your IN is going to be a bit slow, although it might be okay, doing a single table scan into a worktable that Sybase'll index internally. Use an EXISTS instead as well - that might/should work better.
2nd part - both have SARGS (look up in Sybooks if you don't know - search arguments.) I don't know what proportion of rows is found by them but assuming it's a small fraction, you should see an index used on a SARG for whichever table is scanned first, then you should see index join (or perhaps merge join) to the 2nd - but using indexes.
3rd part - similar discussion to the 2nd.
I reckon it'll be the 2nd or 3rd part
How about using cache for these tables. if the query is used kn a regular basis. Its better to get a named cache and bind the tables to it. Also bind the tempdb to cache. This will greatly improve the process execution time. If the temp table is huge then you can create a index on it which may help with performance but i need some more details for that.
If you still have this issue open :
1) Try this at top of sql batch
set showplan on
set noexec on
See if the expected indexes are being picked up by SQL optimizer. If no indexes exist on the columns in where clause, please create one. Create clustered index if possible.
2) In the first query you can replace the subquery in where clause with
create table #T_ID (
M_NB datatype
)
insert into #T_ID
select M_NB from TRN H where (M_BENTITY ="KROP" or M_SENTITY = "KROP")
and modify the where clause as :
where T.M_ID=B.M_ID
and T.T_ID = #T_ID.M_NB
Well, I have a table which is 40,000,000+ records but when I try to execute a simple query, it takes ~3 min to finish execution. Since I am using the same query in my c# solution, which it needs to execute over 100+ times, the overall performance of the solution is deeply hit.
This is the query that I am using in a proc
DECLARE #Id bigint
SELECT #Id = MAX(ExecutionID) from ExecutionLog where TestID=50881
select #Id
Any help to improve the performance would be great. Thanks.
What indexes do you have on the table? It sounds like you don't have anything even close to useful for this particular query, so I'd suggest trying to do:
CREATE INDEX IX_ExecutionLog_TestID ON ExecutionLog (TestID, ExecutionID)
...at the very least. Your query is filtering by TestID, so this needs to be the primary column in the composite index: if you have no indexes on TestID, then SQL Server will resort to scanning the entire table in order to find rows where TestID = 50881.
It may help to think of indexes on SQL tables in the same way as those you'd find in the back of a big book that are hierarchial and multi-level. If you were looking for something, then you'd manually look under 'T' for TestID then there'd be a sub-heading under TestID for ExecutionID. Without an index entry for TestID, you'd have to read through the entire book looking for TestID, then see if there's a mention of ExecutionID with it. This is effectively what SQL Server has to do.
If you don't have any indexes, then you'll find it useful to review all the queries that hit the table, and ensure that one of those indexes is a clustered index (rather than non-clustered).
Try to re-work everything into something that works in a set based manner.
So, for instance, you could write a select statement like this:
;With OrderedLogs as (
Select ExecutionID,TestID,
ROW_NUMBER() OVER (PARTITION BY TestID ORDER By ExecutionID desc) as rn
from ExecutionLog
)
select * from OrderedLogs where rn = 1 and TestID in (50881, 50882, 50883)
This would then find the maximum ExecutionID for 3 different tests simultaneously.
You might need to store that result in a table variable/temp table, but hopefully, instead, you can continue building up a larger, single, query, that processes all of the results in parallel.
This is the sort of processing that SQL is meant to be good at - don't cripple the system by iterating through the TestIDs in your code.
If you need to pass many test IDs into a stored procedure for this sort of query, look at Table Valued Parameters.