I'm trying to figure out a way to optimize my query as it is taking more than 48 hrs to execute the script as I have a huge database. I tried creating all possible indexes in the required tables. I even tried to break the query into sub queries but still no luck with the improve in execution time. Any comments or ideas are appreciated. Here is my query :
SELECT distinct
etb.ID_Etab AS Siret
,ctt.DebutCTT AS DateMouvement
,ctt.L_Contrat_SQN
,ctt.Numero
,ctt.DebutCTT
,ctt_pcs.PcsEse
,ctt.DerDSN
,ctt.H_Etab_SQN
,ctt.H_Salarie_SQN
,ISNULL(ctt_nat.Nature, '') AS Nature,
ctt_dpp.DispPolitiquePublique,
ctt_det_exp.DetacheExpatrie,
ctt_SS.StatutSalarie,
ctt_stat.CodeStatutEmploi,SSS.Num_Inscr AS NIR_RECODE
FROM [dbo].[Lnk_Contrat] AS ctt
INNER JOIN [dbo].[LSat_Contrat_Etab] AS ctt_etb ON ctt.L_Contrat_SQN = ctt_etb.L_Contrat_SQN
INNER JOIN [dbo].[Hub_Etab] AS etb ON ctt_etb.Siret = etb.ID_Etab
INNER JOIN [dbo].[LSat_Contrat_PcsEse] AS ctt_pcs ON ctt.L_Contrat_SQN = ctt_pcs.L_Contrat_SQN and ctt.DebutCTT BETWEEN ctt_pcs.Debut AND ctt_pcs.Fin
INNER JOIN [dbo].[LSat_Contrat_Nature] AS ctt_nat ON ctt.L_Contrat_SQN = ctt_nat.L_Contrat_SQN AND ctt.DebutCTT BETWEEN ctt_nat.Debut AND ctt_nat.Fin AND ISNULL(ctt_nat.Nature, '') <>03
LEFT join Sat_Salarie_SNGI SSS ON SSS.H_Salarie_SQN=ctt.H_Salarie_SQN AND ctt.DebutCTT BETWEEN SSS.Debut AND SSS.Fin
Left JOIN dbo.LSat_Contrat_DispPoliPub AS ctt_dpp ON ctt_dpp.L_Contrat_SQN = ctt.L_Contrat_SQN
INNER JOIN dbo.LSat_Contrat_DetacheExpatrie AS ctt_det_exp ON ctt_det_exp.L_Contrat_SQN = ctt.L_Contrat_SQN AND ctt.DebutCTT BETWEEN ctt_det_exp.Debut AND ctt_det_exp.Fin
INNER JOIN dbo.LSat_Contrat_StatutSalarie AS ctt_SS ON ctt_SS.L_Contrat_SQN = ctt.L_Contrat_SQN AND ctt.DebutCTT BETWEEN ctt_SS.Debut AND ctt_SS.Fin
LEFT JOIN dbo.LSat_Contrat_StatutEmploiSalarie AS ctt_stat ON ctt_stat.L_Contrat_SQN = ctt.L_Contrat_SQN AND ctt.DebutCTT BETWEEN ctt_stat.Debut AND ctt_stat.Fin
WHERE
ctt.Creation_DTS > '1900-01-01'
AND ctt_etb.Debut = '1900-01-01'
AND ctt_etb.[Type] = 'AF'
AND ctt.DebutCTT >= ctt_etb.Debut
AND ctt.DebutCTT >= etb.PreDSN
AND ctt.debutCTT>=DATEADD(MONTH, -1, ctt.PreDSN)
AND ((ctt.DebutCTT BETWEEN ctt_dpp.Debut AND ctt_dpp.Fin) or ctt_dpp.Fin is null)
order by Siret,L_Contrat_SQN
DBCC TRACEOFF (610)
Into the Sql Server Management Studio paste the SQL sentence and click on "Display Estimated Execution Plan". It shows how Sql Server will try to execute your sentence analyzing exiting indexes of your Where(s) and Join(s).
Also try to upgrade your hardware, so each Sql sentence may required a huge ammount of memory and cpu cycles depending of the number of records.
Related
I wrote the below query to pull the data from different databases. I have created two temp tables to pull the data from two different databases and finally a select statement from the original database to join all the tables. My query is getting executed but not getting any data.(Report is blank). I tried executing the two temp tables separately. it is giving the correct data. But when I execute the whole query, the result is blank. Below is the query. Please help.
"set fmtonly off
use GODSDB
IF object_id('tempdb..#CISIS_Call_Log') IS NOT NULL DROP TABLE #CISIS_Call_Log
select *
into #CISIS_Call_Log
from OPENQUERY (CSISDB,
'select
ccl.ContractOID,
ccl.db_insertdate,
ccl.ContractCallLogStatusIdentifier,
ccl.db_UpdateDate,
ccp.ContractCallLogPurposeOID,
ccp.ContractCallLogPurposeIdentifier,
ccp.Description
from csisdb.dbo.ContractCallLog CCL
inner join csisdb.dbo.ContractCallLogPurpose CCP on ccl.ContractCallLogPurposeIdentifier = ccp.ContractCallLogPurposeIdentifier
where JurisdictionShortIdentifier = ''ON''
AND ContractCallLogStatusIdentifier IN (''DNR'', ''NR'')
')
IF object_id('tempdb..#CMS_Campaign') IS NOT NULL DROP TABLE #CMS_Campaign
select *
into #CMS_Campaign
from OPENQUERY (BA_GBASSTOCMS, '
Select
SystemSourceIdentifier,
ContractOID,
OfferSentDate,
CampaignOfferTypeIdentifier,
CampaignContractStatusIdentifier,
CampaignContractStatusUpdateDate,
DeclineDate,
CampaignOfferOID,
CampaignOID,
CampaignStartDate,
CampaignEndDate,
Jurisdiction,
CampaignDescription
from CMS.dbo.vw_CampaignInfo
where Jurisdiction = ''ON''
and CampaignOfferTypeIdentifier = ''REN''
')
select mp.CommodityTypeIdentifier as Commodity
,c.RtlrContractIdentifier as ContractID
,cs.ContractStatusIdentifier as ContractStatus
,c.SigningDate
,cf.StartDate as FlowStartDate
,cf.EndDate as FlowEndDate
,datediff(day, getdate(), c.RenewalDate) as RemainingDays
,c.RenewalDate
,l.ContractCallLogStatusIdentifier as CallLogType
,Substring (l.Description, 1, 20) as CallPurpose
,l.db_insertDate as CallLogDate
,cms.CampaignOfferOID as OfferID
,cms.CampaignContractStatusIdentifier as OfferStatus
,cms.CampaignContractStatusUpdateDate as StatusChangeDate
,cms.DeclineDate
from Contract c
inner join contractstate cs on cs.contractoid = c.ContractOID
and cs.ContractStatusIdentifier in ('ERA', 'FLW')
and datediff(day, getdate(), c.RenewalDate) > 60
inner join SiteIdentification si on si.SiteOID = c.SiteOID
inner join MarketParticipant mp on mp.MarketParticipantOID = si.MarketParticipantOID
inner join Market m on m.MarketOID = mp.MarketOID
inner join Jurisdiction j on j.JurisdictionOID = m.JurisdictionOID
and j.CountryCode = 'CA'
and j.ProvinceOrStateCode = 'ON'
inner join ContractFlow cf on cf.ContractOID = c.ContractOID
inner join #CISIS_Call_Log l on convert(varchar(15), l.ContractOID) = c.RtlrContractIdentifier
inner join #CMS_Campaign cms on convert(varchar(15), cms.ContractOID) = c.RtlrContractIdentifier
set fmtonly on"
IF the data in each temp table is verified, then:
Try a smaller, less complex, query to test your temp tables with. Also try them using a LEFT join as well e.g.:
select
c.RtlrContractIdentifier as ContractID
, c.SigningDate
, datediff(day, getdate(), c.RenewalDate) as RemainingDays
, c.RenewalDate
, l.ContractCallLogStatusIdentifier as CallLogType
, Substring (l.Description, 1, 20) as CallPurpose
, l.db_insertDate as CallLogDate
, cms.CampaignOfferOID as OfferID
, cms.CampaignContractStatusIdentifier as OfferStatus
, cms.CampaignContractStatusUpdateDate as StatusChangeDate
, cms.DeclineDate
from Contract c
LEFT join #CISIS_Call_Log l on convert(varchar(15), l.ContractOID) = c.RtlrContractIdentifier
LEFT join #CMS_Campaign cms on convert(varchar(15), cms.ContractOID) = c.RtlrContractIdentifier
Does this return data? Does it return data from both joined tables?
If neither temp table is returning data then those join conditions need to be changed.
If both temp tables do return data from that query, then try INNER joins. If that still works, then add back more joins (one at a time) until you identify the join that causes the overall fault.
Without data for every table it just isn't possible for us to pinpoint the exact reason for a NULL result. Only you can, so you need to trouble-shoot the problem one step at a time.
We've been using Visual Studio 2017's SQL Server Schema Comparison for all our (SQL Server 2016) migrations and deployments.
However, recently, it has become very slow, taking hours to process. If we uncheck the "Tables" object, it goes quickly. But when tables are checked, it is stuck on "Initializing comparison..." for ages.
I've not been able to find anything online that has helped us. Any ideas?
Initializing comparison...
What seemed to work for us is that if you excluded tables (in Schema Compare Options --> Object types --> Application-scoped -- > Tables), it runs quickly.
After its runs initially, you can compare again with tables selected, and its fine.
With the exception of this, the Visual Studio database Schema Compare is an awesome tool.
You can use MSSQL Server Management Studio's own comparison or use one of these tools:
https://www.agile-code.com/blog/choose-your-sql-server-schema-comparison-tool/
I had very similar problem and I was able to find a solution.
Query starting with SELECT * FROM (SELECT SCHEMA_NAME([o].[schema_id]) AS [SchemaName], from [sys].[spatial_indexes] and other tables had suboptimal plan and was running for hours, causing VS to timeout and retry.
Plan guide included below solved the problem for me.
Because VS sends all queries in one batch during 1st attempt to get results plan guide will not kick in. After query times out, VS will retry this query in its own batch and plan guide will be applied.
To speed up the process even more and don't wait for time out you can kill VS session when it is stuck on this query and force retry.
EXEC sp_create_plan_guide
#name = N'VS Schema Comp Spatial Idxs',
#stmt = N'SELECT * FROM (
SELECT
SCHEMA_NAME([o].[schema_id]) AS [SchemaName],
[si].[object_id] AS [ColumnSourceId],
[o].[name] AS [ColumnSourceName],
[o].[type] AS [ColumnSourceType],
[ic].[column_id] AS [ColumnId],
[c].[name] AS [ColumnName],
[si].[index_id] AS [IndexId],
[si].[name] AS [IndexName],
[ds].[type] AS [DataspaceType],
[ds].[data_space_id] AS [DataspaceId],
[ds].[name] AS [DataspaceName],
[si].[fill_factor] AS [FillFactor],
[si].[is_padded] AS [IsPadded],
[si].[is_disabled] AS [IsDisabled],
[si].[allow_page_locks] AS [DoAllowPageLocks],
[si].[allow_row_locks] AS [DoAllowRowLocks],
[sit].[cells_per_object] AS [CellsPerObject],
[sit].[bounding_box_xmin] AS [XMin],
[sit].[bounding_box_xmax] AS [XMax],
[sit].[bounding_box_ymin] AS [YMin],
[sit].[bounding_box_ymax] AS [YMax],
[sit].[level_1_grid] AS [Level1Grid],
[sit].[level_2_grid] AS [Level2Grid],
[sit].[level_3_grid] AS [Level3Grid],
[sit].[level_4_grid] AS [Level4Grid],
[sit].[tessellation_scheme] AS [TessellationScheme],
[s].[no_recompute] AS [NoRecomputeStatistics],
[p].[data_compression] AS [DataCompressionId],
CONVERT(bit, CASE WHEN [ti].[data_space_id] = [ds].[data_space_id] THEN 1 ELSE 0 END)
AS [EqualsParentDataSpace]
FROM
[sys].[spatial_indexes] AS [si] WITH (NOLOCK)
INNER JOIN [sys].[objects] AS [o] WITH (NOLOCK) ON [si].[object_id] = [o].[object_id]
INNER JOIN [sys].[spatial_index_tessellations] [sit] WITH (NOLOCK) ON [si].[object_id] = [sit].[object_id] AND [si].[index_id] = [sit].[index_id]
INNER JOIN [sys].[data_spaces] AS [ds] WITH (NOLOCK) ON [ds].[data_space_id] = [si].[data_space_id]
INNER JOIN [sys].[index_columns] AS [ic] WITH (NOLOCK) ON [si].[object_id] = [ic].[object_id] AND [si].[index_id] = [ic].[index_id]
INNER JOIN [sys].[columns] AS [c] WITH (NOLOCK) ON [si].[object_id] = [c].[object_id] AND [ic].[column_id] = [c].[column_id]
INNER JOIN [sys].[objects] AS [o2] WITH (NOLOCK) ON [o2].[parent_object_id] = [si].[object_id]
INNER JOIN [sys].[stats] AS [s] WITH (NOLOCK) ON [o2].[object_id] = [s].[object_id] AND [s].[name] = [si].[name]
INNER JOIN [sys].[partitions] AS [p] WITH (NOLOCK) ON [p].[object_id] = [o2].[object_id] AND [p].[partition_number] = 1
LEFT JOIN [sys].[indexes] AS [ti] WITH (NOLOCK) ON [o].[object_id] = [ti].[object_id]
LEFT JOIN [sys].[tables] AS [t] WITH (NOLOCK) ON [t].[object_id] = [si].[object_id]
WHERE [si].[is_hypothetical] = 0
AND [ti].[index_id] < 2
AND OBJECTPROPERTY([o].[object_id], N''IsSystemTable'') = 0
AND ([t].[is_filetable] = 0 OR [t].[is_filetable] IS NULL)
AND ([o].[is_ms_shipped] = 0 AND NOT EXISTS (SELECT *
FROM [sys].[extended_properties]
WHERE [major_id] = [o].[object_id]
AND [minor_id] = 0
AND [class] = 1
AND [name] = N''microsoft_database_tools_support''
))
) AS [_results];
',
#type = N'SQL',
#module_or_batch = NULL,
#params = NULL,
#hints = N'OPTION (FORCE ORDER)';
I am adding some new features to a MS Access DB for a client. The original DB has several queries. The new features that I need to add require that I re-use these queries and incorporate them in my VBA and SQL code.
For example, I am generating a new query (via VBA and SQL) based one of these previous queries. I then export the result as an excel file.
However, whenever I try to run one of the new queries it takes about 15 minutes to complete. During this time there is message in the bottom right of the screen which says "running query. "
Here is one of the SQL queries that I am running. Please note that it ran quickly when there was only one WHERE condition.
SELECT
StudentProgram.fkCohortID AS [Cohort],
Student.pkStudentID AS [Student ID],
Student.EmplID AS [Employee ID],
Student.LastName AS [Last Name],
Student.FirstName AS [First Name],
PostBaccActivity.fkSemesterID AS [Semester],
PostBaccActivity.fkPostBaccID AS [PostBacc],
PostBaccActivity.fkGradSchoolID AS [GradSchool],
PostBaccActivity.ProjectTitle AS [ProjectTitle],
PostBaccActivity.fkFacultyID AS [Faculty],
PostBaccActivity.BeginDate AS [BeginDate],
PostBaccActivity.EndDate AS [EndDate],
PostBaccActivity.Status AS [Status]
FROM qryRptJoinAll
WHERE
qryRptJoinAll.StudentProgram.fkCohortID BETWEEN 1 AND 12
OR qryRptJoinAll.StudentProgram.fkCohortID = 25
OR qryRptJoinAll.StudentProgram.fkCohortID = 28
OR qryRptJoinAll.StudentProgram.fkCohortID = 49
OR qryRptJoinAll.StudentProgram.fkCohortID = 215
OR qryRptJoinAll.StudentProgram.fkCohortID = 220
GROUP BY StudentProgram.fkCohortID, Student.pkStudentID, Student.EmplID, Student.LastName,
Student.FirstName, PostBaccActivity.fkSemesterID, PostBaccActivity.fkPostBaccID,
PostBaccActivity.fkGradSchoolID, PostBaccActivity.ProjectTitle, PostBaccActivity.fkFacultyID,
PostBaccActivity.BeginDate, PostBaccActivity.EndDate, PostBaccActivity.Status
This is the query I am using to generate the other queries:
SELECT
Student.pkStudentID, Student.EmplID, Student.OldID, Student.Inactive,
Student.InactiveReason, Student.Status, Student.LastName, Student.MarriedName,
Student.FirstName, Student.MiddleName, Student.DOB, Student.Sex, Student.SSN,
Student.Email, Student.Race, Student.Ethnicity, Student.EmailSecondary,
Student.fkSemesterBCStart, Student.fkSemesterGrad, Student.TotalCredits,
Student.CreditsAttempted, Student.IndexCredits, Student.QualityPoints,
Student.LocalCredits, Student.TransferCredits, Student.OtherCredits,
StudentProgram.*, StudentEvent.*, StudentResearch.*, StudentEmployment.*,
StudentMajor.*, PostBaccActivity.*, StudentPresentation.*, Presentation.*,
StudentNote.*, SemesterGPA.*, StudentPublication.*, Publication.*, StudentCourse.fkCourseID,
StudentCourse.fkFacultyID, StudentCourse.fkSemesterID, StudentCourse.Grade, Grade.GradeValue,
Grade.NoValue, Course.*, RptCumulativeScienceGPA2.ScienceGPACalc, RptStudentControlList.pkStudentControlID
FROM
((((((((PostBaccActivity RIGHT JOIN ((((((StudentProgram RIGHT JOIN
Student ON StudentProgram.fkStudentID = Student.pkStudentID)
LEFT JOIN Cohort ON StudentProgram.fkCohortID = Cohort.pkCohortID) LEFT JOIN StudentEmployment ON Student.pkStudentID = StudentEmployment.fkStudentID)
LEFT JOIN StudentNote ON Student.pkStudentID = StudentNote.fkStudentID) LEFT JOIN StudentPresentation ON Student.pkStudentID = StudentPresentation.fkStudentID)
LEFT JOIN Presentation ON StudentPresentation.fkPresentationID = Presentation.pkPresentationID) ON PostBaccActivity.fkStudentID = Student.pkStudentID)
LEFT JOIN RptCumulativeScienceGPA2 ON Student.pkStudentID = RptCumulativeScienceGPA2.fkStudentID)
LEFT JOIN RptStudentControlList ON Student.pkStudentID = RptStudentControlList.fkControlID)
LEFT JOIN (StudentPublication LEFT JOIN Publication ON StudentPublication.fkPublicationID = Publication.pkPubID) ON Student.pkStudentID = StudentPublication.fkStudentID)
LEFT JOIN StudentResearch ON Student.pkStudentID = StudentResearch.fkStudentID) LEFT JOIN StudentMajor ON Student.pkStudentID = StudentMajor.fkStudentID)
LEFT JOIN StudentEvent ON Student.pkStudentID = StudentEvent.fkStudentID)
LEFT JOIN (Course RIGHT JOIN (Grade RIGHT JOIN StudentCourse ON Grade.Grade = StudentCourse.Grade) ON Course.pkCourseID = StudentCourse.fkCourseID) ON Student.pkStudentID = StudentCourse.fkStudentID)
LEFT JOIN SemesterGPA ON Student.pkStudentID = SemesterGPA.fkStudentID;
Is there anyway in which I can reduce the time it takes for them to run?
Without knowing the exact queries, can give only generic advice:
add indices to relevant fields in the tables, these depend on the queries
if the same query is repeated, cache the results and reuse them
joins of large tables might be optimized, by pre-filtering and caching
if possible, implement asynchronous queries which can return partial results while still running in the background; this can give the illusion of a faster response
Since you're not using any aggregate functions, get rid of the GROUP BY clause.
Make sure there is an index on StudentProgram.fkCohortID
But my guess is that the actual complexity is in qryRptJoinAll, so you'd have to show us this query too.
This is the schema :
And this is the sql that as I understand is too complex for SQL Optimizer:
SELECT * FROM
(
select pp.Id as PaymentPartId, b.Id as BudgetId, grouping(bp.ID) as g1 , sum(pp.Amount) PaymentsSum, sum(bp.Amount) BudgetSum
from Projects pr
inner join Payments p ON pr.Id = p.ProjectID
inner join PaymentParts pp ON p.Id = pp.PaymentId
inner join Budgets b ON pr.Id = b.ProjectID
inner join Budgetparts bp ON b.Id = bp.BudgetId
group by pp.Id, b.Id, rollup(bp.ID)
) x
WHERE x.PaymentPartId = 777
SQLFIDDLE: http://sqlfiddle.com/#!6/aa74e/11 (with autogenerated data)
What I expect: execution plan should contain index seek on x.PaymentPartId. Why? Because this query is equivalent to:
select pp.Id as PaymentPartId, b.Id as BudgetId, grouping(bp.ID) as g1, sum(pp.Amount) PaymentsSum, sum(bp.Amount) BudgetSum
from Projects pr
inner join Payments p ON pr.Id = p.ProjectID
inner join PaymentParts pp ON p.Id = pp.PaymentId
inner join Budgets b ON pr.Id = b.ProjectID
inner join Budgetparts bp ON b.Id = bp.BudgetId
WHERE pp.Id = 777
group by pp.Id, b.Id, rollup(bp.ID)
...and the last query uses index seek.
But SQL Optimizer not only refuse to use the index but ignore all hints (I propose you to expirement wiht sqlfiddle - it is really interesting).
So the question is: am I right that it is impossible to force SQL Server Optimizer to use index seek there? It seems like rollup is something that split sql optimizer "optimization frame" to two parts and it makes it impossible to optimize WHOLE query.
P.S. For those who votes for closing this "non-programming question": try to put optimizer hints (sqlfiddle is ready to test your programming skills!).
Why hints doesn't work? - Roman Pokrovskij
It is in the documentation:
http://technet.microsoft.com/en-us/library/ms181714.aspx
Query hints can be specified only in the top-level query, not in subqueries
I have following query and it execute just fine.
It takes less than 1 sec to get results.
SELECT DISTINCT Items.ImageID AS ImgID, Items.AddDate, MIN(ra1.DescPriority) AS Pri
FROM Items
INNER JOIN Attribs AS ra0 ON Items.ImageID = ra0.ImageID
INNER JOIN Attribs AS ra1 ON ra0.ImageID = ra1.ImageID
LEFT OUTER JOIN v_priceOrder ON Items.ImageID = v_priceOrder.ImageID
WHERE (Items.deleted NOT IN (1, 2))
GROUP BY Items.ImageID, Items.AddDate
ORDER BY pri, Items.AddDate DESC
However, if I try to add a column from PriceOrder View in select lists it takes 10 seconds to get results.
SELECT DISTINCT Items.ImageID AS ImgID, Items.AddDate, MIN(ra1.DescPriority) AS Pri,
v_priceOrder.PriceOrder
FROM Items
INNER JOIN Attribs AS ra0 ON Items.ImageID = ra0.ImageID
INNER JOIN Attribs AS ra1 ON ra0.ImageID = ra1.ImageID
LEFT OUTER JOIN v_priceOrder ON Items.ImageID = v_priceOrder.ImageID
WHERE (Items.deleted NOT IN (1, 2))
GROUP BY Items.ImageID, Items.AddDate, v_priceOrder.PriceOrder
ORDER BY v_priceOrder.PriceOrder, pri, Items.AddDate DESC
Is there a way to improve this query?
In the Execution plan I only see 0 or 1% cost.
I'm using MS SQL Server 2008 R2.
UPDATE:
My View had a issue. It takes 8 seconds to run...
Here is the Query.
SELECT dbo.Items.ImageID,
CASE WHEN dbo.Items.SubcategoryID2 = 'STO' THEN dbo.v_DisplayStockPrice.DisplayPrice WHEN dbo.Items.SubcategoryID2 = 'ORD' THEN dbo.v_FinalPriceFactor.[14kyFactorOrder]
WHEN dbo.Items.SubcategoryID2 IS NULL THEN dbo.v_FinalPriceFactor.[14kyCustomLow] END AS PriceOrder
FROM dbo.v_FinalPriceFactor RIGHT OUTER JOIN
dbo.Items ON dbo.v_FinalPriceFactor.ImageID = dbo.Items.ImageID LEFT OUTER JOIN
dbo.v_DisplayStockPrice ON dbo.Items.ImageID = dbo.v_DisplayStockPrice.ImageID
WHERE (dbo.Items.deleted NOT IN (1, 2))
I had cross join in my Views causing unnecessary loops...