Related
I'm need to run a calculation for month every day. If the month period, exists already, I need to update it, else I need to create a new row for the new month.
Currently, I've written
declare #period varchar(4) = '0218'
DECLARE #Timestamp date = GetDate()
IF EXISTS(select * from #output where period=#period)
/* UPDATE #output SET --- same calculation as below ---*/
ELSE
SELECT
#period AS period,
SUM(timecard.tworkdol) AS dol_local,
SUM(timecard.tworkdol/currates.cdrate) AS dol_USD,
SUM(timecard.tworkhrs) AS hrs,
#Timestamp AS timestamp
FROM dbo.timecard AS timecard
INNER JOIN dbo.timekeep ON timecard.ttk = timekeep.tkinit
INNER JOIN dbo.matter with (nolock) on timecard.tmatter = matter.mmatter
LEFT JOIN dbo.currates with (nolock) on matter.mcurrency = currates.curcode
AND currates.trtype = 'A'
AND timecard.tworkdt BETWEEN currates.cddate1
AND currates.cddate2
WHERE timekeep.tkloc IN('06','07') AND
timecard.twoper = #period
SELECT * FROM #output;
How can simply update my row with the new data from my select.
Not sure what RDBMS are you using, but in SQL Server something like this would update the #output table with the results of the SELECT that you placed in the ELSE part:
UPDATE o
SET o.dol_local = SUM(timecard.tworkdol),
SET o.dol_USD = SUM(timecard.tworkdol/currates.cdrate),
SET o.hrs = SUM(timecard.tworkhrs),
set o.timestamp = #Timestamp
FROM #output o
INNER JOIN dbo.timecard AS timecard ON o.period = timecard.twoper
INNER JOIN dbo.timekeep ON timecard.ttk = timekeep.tkinit
INNER JOIN dbo.matter with (nolock) on timecard.tmatter = matter.mmatter
LEFT JOIN dbo.currates with (nolock) on matter.mcurrency = currates.curcode
AND currates.trtype = 'A'
AND timecard.tworkdt BETWEEN currates.cddate1
AND currates.cddate2
WHERE timekeep.tkloc IN('06','07') AND
timecard.twoper = #period
Also, I think you want to do an INSERT in the ELSE part, but you are doing just a SELECT, so I guess you should fix that too
The answer to this will vary by SQL dialect, but the two main approaches are:
1. Upsert (if your DBMS supports it), for example using a MERGE statement in SQL Server.
2. Base your SQL on an IF:
IF NOT EXISTS (criteria for dupes)
INSERT INTO (logic to insert)
ELSE
UPDATE (logic to update)
DECLARE #U_CHK TABLE (
ACCOUNT_ID NUMERIC
,START_NO NUMERIC
,END_NO NUMERIC
,SEQ_NO NUMERIC
,USED INT
)
DECLARE #ST_NO NUMERIC
,#END_NO NUMERIC
,#ACID NUMERIC
,#SEQ_NO NUMERIC
DECLARE cCurChk Scroll CURSOR
FOR
SELECT DISTINCT AC_ID
,ST_DOC_NO
,ST_DOC_NO
,END_DOC_NO
FROM AC_LVL_INVEN
GROUP BY
ST_DOC_NO
,END_DOC_NO
,AC_ID
OPEN cCurChk
FETCH NEXT FROM cCurChk INTO #ACID,#ST_NO,#SEQ_NO,#END_NO
WHILE ##Fetch_Status=0
BEGIN
WHILE (#SEQ_NO<=#END_NO)
BEGIN
INSERT INTO #U_CHK
VALUES
(
#ACID
,#ST_NO
,#END_NO
,#SEQ_NO
,0
)
SET #SEQ_NO = #SEQ_NO+1
END FETCH NEXT FROM cCurChk
INTO #ACID,#ST_NO,#SEQ_NO,#END_NO
END CLOSE cCurChk DEALLOCATE cCurChk
UpDate #U_CHK
SET USED = 1 FROM #U_CHK Ch INNER JOIN FA_Trans FA(NOLOCK) ON Fa.
Account_ID = Ch.Account_ID AND CONVERT(VARCHAR(10) ,Ch.Seq_No) = Fa.Instrument_No
WHERE FA.Status>4 AND STATUS<>12 AND Ac_Head_Type<>1 AND Trans_type = 'Debit'
You can use a merge statement to do the insert / update in place of the cursor. Here is an example
MERGE dbo.FactBuyingHabits AS Target
USING (SELECT CustomerID, ProductID, PurchaseDate FROM dbo.Purchases) AS Source
ON (Target.ProductID = Source.ProductID AND Target.CustomerID = Source.CustomerID)
WHEN MATCHED THEN
UPDATE SET Target.LastPurchaseDate = Source.PurchaseDate
WHEN NOT MATCHED BY TARGET THEN
INSERT (CustomerID, ProductID, LastPurchaseDate)
VALUES (Source.CustomerID, Source.ProductID, Source.PurchaseDate)
Take a look at Technet - Inserting, Updating, and Deleting Data by Using MERGE
I have a VERY long running, heavy stored procedure, so I would like to change it over to a job that runs every night and updates a table so the user can run it quickly during the day. My problem is that it takes parameters IN THE LEFT JOIN, so I don't see how I could do it. I tried to remove the parameters from the joins and just take all the records, but first problem with that is that these left joins are joins to table valued functions with group by clauses (so it's tons of records now) and also when I did it, it crashed my whole server. I will post part of my stored proc to give you an idea of what i'm doing for now.
--alternatively, i would take an answer that could change my query perfomance from 2 minutes to 10 seconds. :)
--btw, another thing that is slowing down this query is that the functions are querying a linked server...
create PROC [dbo].[mySP]
#FromDate DateTime,
#ToDate DateTime = NULL,
#Style NVARCHAR(max),
#ItemType NVARCHAR(300),
#ItemCode NVARCHAR(150)= null,
#ItemsNotSold INT,
#MultiplyUsage DECIMAL(18,4),
#SParts BIT,
#vendorId nvarchar(50),
#currentPage int,
#largeOrderSoNumbers nvarchar(300) = null,
#excludeprojTag BIT,
#hasOrderQty BIT = null,
#createPO BIT = null,
#excludeTestOrders BIT = null,
#refresh BIT = null,
#userid int = null,
#positiveOverUse BIT = null
AS
SELECT #FROMDATE = #FROMDATE
,#ToDate=#ToDate
,#Style=#Style
,#ItemType=#ItemType
,#ItemCode=#ItemCode
,#ItemsNotSold=#ItemsNotSold
,#MultiplyUsage=#MultiplyUsage
,#SParts=#SParts
,#vendorId=#vendorId
,#currentPage=#currentPage
,#largeOrderSoNumbers=#largeOrderSoNumbers
,#excludeprojTag=#excludeprojTag
,#hasOrderQty =#hasOrderQty
,#createPO =#createPO
,#excludeTestOrders =#excludeTestOrders
,#refresh =#refresh
,#userid =#userid
,#positiveOverUse=#positiveOverUse
DECLARE #amountPerPage int = 400;
DECLARE #startingPoint int = (#currentpage - 1) * #amountPerPage
SET #TODATE = NULLIF(#TODATE,'1/1/4000');
SET #ITEMCODE = NULLIF(#ITEMCODE,'');
SET #largeOrderSoNumbers = NULLIF(#LARGEORDERSONUMBERS,'');
BEGIN
IF #Style = '-1'
SET #Style = NULL
IF #ItemType = '-1'
SET #ItemType = NULL
END
SELECT
ISNULL(c.NewQB_ListID,iit.ListID) ListID,
(select salesdesc from iteminventory i where i.listid = ISNULL(c.NewQB_ListID,iit.ListID)) as SalesDesc,
isnull(inl.linename,'') + isnull(c.NewName,Name) as ItemCode ,
max(il.CubicMeterKD) as CubicMeterKD,
sum(CONVERT(decimal(18,4),inl.Quantity) )
- sum(CONVERT(decimal(18,4),inl.qtyRuleTag))
+ CONVERT(decimal(18,4),isnull(ai.qty,0))
-CONVERT(decimal(18,4),isnull(ai.QtyRuleTag,0))
AS TotalQtyUesd,
sum(convert(decimal(18,4),isnull(inl.QtyRuleTag,0)))
as QtyUsedWithProjectTag,
Convert(decimal(18,4),QuantityOnHand) as OnHand,
CONVERT(int,ISNULL(EzQtyOnSO.Qty,0)) as OnSalesOrder,
CONVERT(int,ISNULL(EzQtyAnyStatus.QtyLgO,0))as OnLgOrder,
Convert(decimal(18,4),QuantityOnOrder) as OnPO,
Convert(decimal(18,4),isnull(LargePO.lgOrderQtyOnPO,0)) as lgOrderQtyOnPO ,
fl.LineName
INTO #Q
FROM
iteminventory iit
LEFT JOIN Lines fl
ON iit.ParentRef_ListID = QB_LisiID
LEFT JOIN tb_ItemList il
ON iit.ListID = il.QBListID
LEFT JOIN InlSales(#FromDate,#ToDate, #excludeprojtag, #excludeTestOrders) INL
ON IIT.LISTID = INL.ITEMREF_LISTID
LEFT JOIN tb_CombinedItems c
ON iit.ListID = c.ListID
LEFT JOIN [QuantityInvoice](#FromDate, #ToDate,#excludeprojTag,#excludeTestOrders) as ai
ON ai.QBID = iit.ListID
LEFT JOIN fn_QuantityOnSalesOrder(#excludeTestOrders) AS EzQtyOnSO
ON iit.ListID = EzQtyOnSO.QBID
LEFT JOIN QuantityAnyStatus(#largeOrderSoNumbers, #excludeprojTag) AS EzQtyAnyStatus
ON iit.ListID = EzQtyAnyStatus.QBID
LEFT JOIN dbo.[FN_LargePO](#excludeTestOrders) LargePO --WITH (nolock)
ON LargePO.QBID = iit.ListID
WHERE
(#Style is null or ISNULL(c.ParentListID,ParentRef_ListID) in (SELECT Value From fn_MultiValueParameter(#Style))) AND
(#ItemType is null or ItemTypeCode in (SELECT Value From fn_MultiValueParameter(#ItemType))) AND
(#ItemCode is null or il.ItemCode like '%' + #ItemCode + '%') AND
(IsActive = 1 OR c.ListID IS NOT NULL)
GROUP BY
ISNULL(c.NewQB_ListID,iit.ListID),
ISNULL(c.NewName,Name),
inl.LineName,
QuantityOnHand,
iit.QuantityOnOrder,
CONVERT(int,ISNULL(EzQtyOnSO.Qty,0)),
fl.LineName,
CONVERT(decimal(18,4),isnull(ai.qty,0)),
Convert(decimal(18,4),isnull(LargePO.lgOrderQtyOnPO,0)),
CONVERT(int,ISNULL(EzQtyAnyStatus.QtyLgO,0)),
CONVERT(decimal(18,4),isnull(ai.QtyRuleTag,0))
IF #ItemsNotSold = 1
BEGIN
DELETE FROM #Q
WHERE isnull(TotalQtyUesd,0) <> 0
END
IF #ItemsNotSold = 2
BEGIN
DELETE FROM #Q
WHERE isnull(TotalQtyUesd,0) = 0
END
SELECT
q.ListID,
q.SalesDesc,
q.ItemCode,
MAX(isnull(q.CubicMeterKD,0)) as CubicMeterKD,
SUM(q.TotalQtyUesd)
TotalQtyUesd,
SUM(q.QtyUsedWithProjectTag) as QtyUsedWithProjectTag,
SUM(q.OnHand) OnHand,
SUM(q.OnSalesOrder) OnSalesOrder,
SUM(q.OnLgOrder) OnLgOrder,
SUM(q.OnPO)
- isnull(SUM(q.lgOrderQtyOnPO),0)
OnPO,
SUM(q.lgOrderQtyOnPO) as lgOrderQtyOnPO,
LineName
INTO #QTY
FROM
#Q q
GROUP BY
q.ListID,
q.salesdesc,
q.ItemCode,
q.LineName
;WITH Results AS
(
SELECT
isnull(P.listid,'') as ListID,
isnull(P.SalesDesc,'') as SalesDesc,
isnull(P.itemcode,'') as ItemCode,
ISNULL(CubicMeterKD,0) AS CubicMeterKD,
isnull(TotalQtyUesd,0) as TotalQtyUsed,
isnull(QtyUsedWithProjectTag,0) as QtyUsedWithProjectTag,
isnull(onhand,0) as OnHand,
isnull((OnHand - OnSalesOrder - OnLgOrder),0) as available,
isnull(OnLgOrder,0) as OnLgOrder,
isnull(OnPO,0) as OnPo,
isnull(lgOrderQtyOnPO,0) as lgOrderQtyOnPO,
isnull(((OnHand - OnSalesOrder) + OnPO) ,0) AS [AvailableAndOnPo],
isnull((TotalQtyUesd * #MultiplyUsage) - ((OnHand - OnSalesOrder) + OnPO),0) AS [AvlOnPOminusUsed],
isnull((((TotalQtyUesd * #MultiplyUsage) - ((OnHand - OnSalesOrder) + OnPO)) / CASE WHEN TotalQtyUesd > 0 THEN TotalQtyUesd END)*100,0) AS PctOver
,isnull(linename,0) as LineName,
isnull((select price from qbdb.dbo.tb_iteminfodetail where vendorlistid = #vendorid and itemcode =isnull(P.itemcode,'')),0.0) as price
,isnull(pod.qtytoReOrder,0) as qtyToReOrder
,isnull(pod.qtytoOrder,0) as qtyToOrder
,isnull(pod.includePO,0) as includePO
,ROW_NUMBER() OVER (ORDER BY ITEMCODE) AS RowNum
FROM #qty AS p
left join dbo.purchaseorderpreliminarydetails pod on pod.listid = P.listid and pod.deleteFlag = 0
and headerid = (select top 1 headerid from purchaseorderpreliminary where deleteFlag = 0)
WHERE
CASE WHEN #positiveOverUse = 1 THEN
isnull((((TotalQtyUesd * #MultiplyUsage) - ((OnHand - OnSalesOrder) + OnPO)) / CASE WHEN TotalQtyUesd > 0 THEN TotalQtyUesd END)*100,0)
ELSE 1 END >0
AND
case when #hasOrderQty = 1 then isnull(pod.QtyToReOrder,0) else 1 end > 0
and
CASE WHEN #SPARTS = 1 THEN CASE WHEN
ItemCode IN (
'APP',
'CD',
'-S',
'L0',
'L/42',
'L01',
'Lfrgs2',
'Lfad2',
'Sfasdf9',
'SdafdsA',
'Sfasdf3',
'Sasdf6',
'asdf0',
'Sf6',
'fasdfadf2',
'fasdfasdf',
'S2236',
'S12342',
'Sdf 30',
'SdfE 36',
'fgsfgs',
'fasdf-fdasf',
'fadf',
'fasdf-fasdf',
'sdaf',
'adf 11"',
'fda 14"',
'fdas 24"') THEN 1 ELSE 0 END ELSE 1 END = 1
)
SELECT (select count(rownum) from results) as totalItems,* FROM Results WHERE
rownum between #startingPoint + 1 and #startingPoint + #amountPerPage
Have a form the users can put their parameters into before they go home, and save them into a new table in the database. Change the stored procedure to either get the parameters from this new table or to use the new table in the joins &c. The SP now doesn't need the parameters to sent to it. Finally call the SP via a Job.
I need to improve the performance of a stored procedure that is used by SSRS. When a very narrow set of parameters are passed, the stored procedure completes rather quickly. But when more than a hundred thousand results are returned, the stored procedure takes minutes to run.
For example, if I run this using the date range of 1982 - 2020, it takes 15 minutes to run and returns 180000 records.
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
if exists (select * from dbo.sysobjects where id = object_id(N'[dbo].[uspContractFundingbyYear2]') and OBJECTPROPERTY(id, N'IsProcedure') = 1)
drop procedure [dbo].[uspContractFundingbyYear2]
GO
-- =============================================
-- Author: J. Dickens
-- Create date: April 2011
-- Description: This SP generates contract funding records for the selected parameters.
-- Each row should represent a single contract funding record.
-- In many cases, either 0 or 100 is used to designate an option that is not selected. The parameter should have no impact on the query.
-- Otherwise all values are passed to the parameter using the multivalue parameter function.
-- =============================================
CREATE PROCEDURE [dbo].[uspContractFundingbyYear2]
#StartYear datetime
,#EndYear datetime
,#BusinessEntityID nvarchar(max)
,#ContractTypeCode nvarchar(max)
,#ContractStatusCode nvarchar(max)
,#ContractOrgID nvarchar(max)
,#ResearchTypeCode nvarchar(max)
,#ExportControlLevelCode nvarchar(max)
,#ScienceAreaID nvarchar(max)
,#ThrustAreaID nvarchar (max)
,#ContractStartdate datetime
,#ContractEnddate datetime
,#ResearchBusID int
,#MasterContractOnlyFlag bit
,#StateProvinceCode nvarchar(max)
,#CountryID nvarchar(max)
,#FinalYearFlag int
,#CollaborativeDollars int
,#SubcontractsFlag int
,#FundingOrgID nvarchar(max)
,#FundingTypeCode nvarchar(max)
,#FundingStatusCode nvarchar(max)
,#FundingStartdate datetime
,#FundingEnddate datetime
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
--- Prepare a table of Year Numbers, for use with Cross Join
DECLARE #Dates TABLE (YearNo INT)
;WITH n(n) AS
(
SELECT YEAR(#StartYear)
UNION ALL
SELECT n+1
FROM n
WHERE n < YEAR(#EndYear)
)
INSERT INTO #Dates(YearNo)
SELECT n
FROM n
ORDER BY n
-- Now find all the contract funding records
select distinct
be.BusinessEntityName
, Q.ScienceArea
, r.ResearchBusID
, c.FormattedBusID
, o.Abbreviation as 'ContractOrg'
, c.StateProvinceCode
, cy.CountryName
, cf.ContractFundingID
, c.ContractTypeCode
, c.ContractID,
, [Thrust] = dbo.ufConcatenateThrustsforResTaskDate (r.ResearchID, NULL, NULL, NULL)
, [PI] = (select STUFF((select distinct(coalesce('; '+P.lastfirst,''))
from ResTaskParticipant rtp
join PersonOrg po on rtp.PersonOrgID = po.PersonOrgID
join Person p on po.PersonID = p.PersonID
where rtp.ResearchID = r.researchid
and rtp.ParticipantTypeCode = 'PI'
and (rtp.enddate > getdate() or rtp.enddate = r.enddate)
for XML path ('')),1,2,'')),
c.ContractStatusCode,
cf.Fundingstatuscode as 'FundingStatus', cf.FundingTypeCode as 'FundingType', cf.Funding,
o2.Abbreviation as 'FundingSourceOrg', cf.StartDate, cf.EndDate,
DATEDIFF(m,cf.StartDate, dateadd(d,2,cf.EndDate)) as 'ContractMonths', --using 2 days here to allow for leap years
[BurnRate] =
case
when DATEDIFF(m,cf.StartDate, dateadd(d,2,cf.EndDate)) = 0 --using 2 days here to allow for leap years
then convert(decimal(2,1), 0)
else
convert(decimal (15,5), convert(decimal(15,0),cf.Funding)/convert(decimal(2,0),(DATEDIFF(m,cf.StartDate, dateadd(d,2,cf.EndDate))))) --using 2 days here to allow for leap years
end
, [IsFirstOfMonth] =
case when cf.startdate = (select FirstDayofMonth from dbo.ufGetFirstLastDayofMonth(cf.StartDate, 0)) then 1 else 0 end
, [IsEndOfMonth] =
case when datepart(day,cf.EndDate) = datepart(day,(select LastDayofMonth from dbo.ufGetFirstLastDayofMonth(cf.EndDate, 0))) then 1 else 0 end
, [MonthsInYear] = Convert(decimal(2,0), (select dbo.ufmonthsincontractforyear (cf.startdate, cf.EndDate, d.YearNo)))
,[YearNo] = d.YearNo
from ContractFunding cf
join Contract c on cf.ContractID = c.ContractID
join Research r on c.ResearchID = r.ResearchID
join BusinessEntity be on r.BusinessEntityID = be.BusinessEntityID
join Organization o on c.OrganizationID = o.OrganizationID --Contract Org
join Country cy on c.CountryID = cy.CountryID
join Organization o2 on cf.SourceOrganizationID = o2.OrganizationID --Funding Org
left outer join TaskFunding tf on cf.ContractFundingID = tf.ContractFundingID
--Get the range of years for each Contract Funding row
cross join #Dates d
--get Science Area
cross apply (select [ScienceArea] = dbo.ufConcatenateScienceAreasforResTaskDate (r.ResearchID, NULL, NULL, NULL)) as Q
where
-- required parameters for running report are Start and End Years
cf.StartDate <= #EndYear and cf.EndDate >= #StartYear
-- now all the bells and whistles
and r.BusinessEntityID in (select val from [dbo].[ufStringToTable] (#BusinessEntityID,',',1))
and c.ContractTypeCode in (select val from [dbo].[ufStringToTable] (#ContractTypeCode,',',1))
and c.ContractStatusCode in (select val from [dbo].[ufStringToTable] (#ContractStatusCode,',',1))
and c.OrganizationID in (select val from dbo.ufStringToTable (#ContractOrgID,',',1))
and (#ResearchTypeCode = '0' or (r.ResearchTypeCode in (select val from dbo.ufStringToTable (#ResearchTypeCode,',',1))))
and (#ExportControlLevelCode = '100' or c.ExportControlLevelCode = #ExportControlLevelCode)
-- For Science Area Funding, of the set of Contract Funding records returned, select those that match the given Science Area
and (#ScienceAreaID = '0' or (cf.ScienceAreaID in (select val from dbo.ufStringToTable (#ScienceAreaID,',',1))))
-- For Thrust Area Funding, of the set of Contract Funding records returned, select those that match the given Thrust Area
and (#ThrustAreaID = '0' or (tf.ThrustAreaID in (select val from dbo.ufStringToTable (#ThrustAreaID,',',1))))
-- Find Contracts within given "Active" dates
and (
(#ContractStartdate is null and #ContractEnddate is null)
OR
(c.enddate > #ContractStartdate and c.StartDate <=#ContractEnddate)
or
(#ContractStartdate is null and c.StartDate <=#ContractEnddate)
or
(c.EndDate > #ContractStartdate and #ContractEnddate is null)
)
and (#ResearchBusID = '' or r.ResearchBusID = #ResearchBusID)
and (#MasterContractOnlyFlag = 0 or (#MasterContractOnlyFlag = 1 and c.MasterContractFlag = 1))
and (#StateProvinceCode = '0' or (c.StateProvinceCode in (select val from dbo.ufStringToTable (#StateProvinceCode,',',1))))
and c.CountryID in (select val from dbo.ufStringToTable (#CountryID,',',1)) --Not all contracts have a country assignment
and (#FinalYearFlag = 100
or
(#FinalYearFlag = 0 and (dbo.ufContractIsInFinalYear(c.contractid,getdate()) = 0))
or
(#FinalYearFlag = 1 and (dbo.ufContractIsInFinalYear(c.contractid,GETDATE()) = 1))
)
and (#CollaborativeDollars = 100
or
(#CollaborativeDollars = 0 and
not exists
(select * from ContractFunding cf2 where cf2.FundingTypeCode='Collaborative' and cf2.ContractID=c.ContractID)
)
or
(#CollaborativeDollars = 1 and
exists
(select * from ContractFunding cf2 where cf2.FundingTypeCode='Collaborative' and cf2.ContractID=c.ContractID)
)
)
and (#SubcontractsFlag = 100
or
(#SubcontractsFlag = 0 and c.SubContractFlag = 0)
or
(#SubcontractsFlag = 1 and c.SubContractFlag = 1)
)
and (cf.SourceOrganizationID in (select val from dbo.ufStringToTable (#FundingOrgID,',',1)))
and (cf.FundingTypeCode in (select val from dbo.ufStringToTable (#FundingTypeCode,',',1)))
and (cf.FundingStatusCode in (select val from dbo.ufStringToTable (#FundingStatusCode,',',1)))
and (
(#FundingStartdate is null and #FundingEnddate is null)
OR
(c.enddate > #FundingStartdate and c.StartDate <=#FundingEnddate)
or
(#FundingStartdate is null and c.StartDate <=#FundingEnddate)
or
(c.EndDate > #FundingStartdate and #FundingEnddate is null)
)
End
GO
Our DBA has changed a function to be a procedure, so I am amending some of my procedures to cater for this. I've come across a problem with this in one of my procedures where I have a while loop.
I populate my temp table from the new procedure (#DGContol), and then have the following while loop:
SELECT #MinRcd = MIN(RcdNum) FROM #PortfolioDisclosure
SELECT #MaxRcd = MAX(RcdNum) FROM #PortfolioDisclosure
SET #RcdNum = #MinRcd
WHILE #RcdNum <= #MaxRcd
BEGIN
-- Temporarily assign values to variables
SELECT
#PortfolioGroup = PortfolioCode
, #EndDateTime = MaxPositionDate_DetailedDisclosure
FROM #PortfolioDisclosure
WHERE RcdNum = #RcdNum
INSERT INTO #PositionsTable
SELECT
fi.ID_ISIN AS [Fund_ISIN]
, RTRIM(a.acct_id) AS [Internal_Portfolio_Code]
, a.acct_desc AS [Portfolio/Fund_Name]
, CONVERT(CHAR(11),p.as_of_tms,103) AS [Portfolio_Date]
, a.alt_curr_cde AS [Portfolio_Base_Ccy]
, i.iss_desc AS [Security_Description]
, RTRIM(i.pref_iss_id) AS [Security_ID SEDOL/Internal]
, RTRIM(ia.iss_id) AS [Security_ID ISIN]
, i.denom_curr_cde AS [Denomination_Ccy]
, SUM(p.valval_alt_cmb_amt) OVER (PARTITION BY RTRIM(a.acct_id))
AS [Total_Fund_Value]
, p.orig_quantity AS [Shares/Par_Value]
, p.valval_alt_cmb_amt AS [Market_Value]
, p.fld5_rte AS [Pct_of_NAV]
, SUM(CASE WHEN i.issue_cls1_cde = '010' THEN p.valval_alt_cmb_amt ELSE 0 END) OVER (PARTITION BY a.acct_id)
AS [Cash/Cash_Equivalents]
, i.inc_proj_cmb_rte AS [Coupon_Rate]
, CONVERT(CHAR(11),i.mat_exp_dte,103) AS [Maturity_Date]
FROM dw_position_dg AS p
INNER JOIN #DGControl AS dgc -- [M]onthly, [M]ost recent position
ON dgc.DataGrpCtlNum = p.data_grp_ctl_num
INNER JOIN dw_ivw_acct AS a WITH (NOLOCK)
ON a.acct_id = p.acct_id
INNER JOIN dw_issue_dg AS i WITH (NOLOCK)
ON i.instr_id = p.instr_id
LEFT OUTER JOIN dw_issue_alt_id AS ia WITH (NOLOCK)
ON ia.instr_id = i.instr_id
AND ia.id_ctxt_typ = 'ISIN'
INNER JOIN #PortfolioDisclosure AS fi
ON fi.PortfolioCode = p.acct_id
and fi.MaxPositionDate_DetailedDisclosure >= p.as_of_tms
WHERE RTRIM(a.acct_id) NOT IN ( SELECT xref.internal_value FROM dbo.DP_CrossReference as xref
WHERE xref.Codeset_type_id = 10401
AND xref.Originator_id = 'DataVendorPortfolioExclusion')
-- Clear down variable values
SET #PortfolioGroup = NULL
--SET #StartDateTime = NULL
SET #EndDateTime = NULL
-- Move to next record
SET #RcdNum = #RcdNum + 1
END -- END WHILE LOOP
However this returns lots of duplicate records. If I replace the temp table #DGControl with the original function then I get the correct number of records.
I don't really know what the issue would be or how I could re code this while loop so that using the table #DGControl I get the correct number of records. Can anyone help?
Have you compared the output of SELECT * FROM OldFunction(args..) with EXEC NewStoredProcedure args...? If so, does the data returned look the same or has the duplication crept in when the DBA refactored the function to a proc.
If so you may need to de dupe the output from the sp first then run it through your remaining code. In short, if you are using the same arguments for both but they give you different results then you'll need to go back to the DBA.
You aren't using either of your local variables (#PortfolioGroup, #EndDateTime) in this code. It is just inserting the same record set N number of times. Also, I think you can write this as a single select query without using temp tables or while loops and it will make it less confusing.
Thank you for your feedback. i worked out the issue. My code looks like the following now:
BEGIN
SELECT #MinRcd = MIN(RcdNum) FROM #PortfolioDisclosure
SELECT #MaxRcd = MAX(RcdNum) FROM #PortfolioDisclosure
SET #RcdNum = #MinRcd
WHILE #RcdNum <= #MaxRcd
BEGIN
-- Temporarily assign values to variables
SELECT
#PortfolioGroup = PortfolioCode
, #EndDateTime = MaxPositionDate_DetailedDisclosure
FROM #PortfolioDisclosure
WHERE RcdNum = #RcdNum
-- Insert DGControl record into table based on the MaxPositionDate_DetailedDisclosure from Portfolio Disclosure function
INSERT INTO #DGControl
EXEC InfoPortal..usp_Generic_DGControl '', '', #PortfolioGroup, '1', #EndDateTime, #EndDateTime, #PeriodType, 'M', 'POSITION' -- [M]onthly, [M]ost recent position
-- Insert into #PositionsTable
INSERT INTO #PositionsTable
SELECT
fi.ID_ISIN AS [Fund_ISIN]
, RTRIM(a.acct_id) AS [Internal_Portfolio_Code]
, a.acct_desc AS [Portfolio/Fund_Name]
, CONVERT(CHAR(11),p.as_of_tms,103) AS [Portfolio_Date]
, a.alt_curr_cde AS [Portfolio_Base_Ccy]
, i.iss_desc AS [Security_Description]
, RTRIM(i.pref_iss_id) AS [Security_ID SEDOL/Internal]
, RTRIM(ia.iss_id) AS [Security_ID ISIN]
, i.denom_curr_cde AS [Denomination_Ccy]
, SUM(p.valval_alt_cmb_amt) OVER (PARTITION BY RTRIM(a.acct_id))
AS [Total_Fund_Value]
, p.orig_quantity AS [Shares/Par_Value]
, p.valval_alt_cmb_amt AS [Market_Value]
, p.fld5_rte AS [Pct_of_NAV]
, SUM(CASE WHEN i.issue_cls1_cde = '010' THEN p.valval_alt_cmb_amt ELSE 0 END) OVER (PARTITION BY a.acct_id)
AS [Cash/Cash_Equivalents]
, i.inc_proj_cmb_rte AS [Coupon_Rate]
, CONVERT(CHAR(11),i.mat_exp_dte,103) AS [Maturity_Date]
FROM dw_position_dg AS p
INNER JOIN #DGControl AS dgc
ON dgc.DataGrpCtlNum = p.data_grp_ctl_num
INNER JOIN dw_ivw_acct AS a WITH (NOLOCK)
ON a.acct_id = p.acct_id
INNER JOIN dw_issue_dg AS i WITH (NOLOCK)
ON i.instr_id = p.instr_id
LEFT OUTER JOIN dw_issue_alt_id AS ia WITH (NOLOCK)
ON ia.instr_id = i.instr_id
AND ia.id_ctxt_typ = 'ISIN'
INNER JOIN #PortfolioDisclosure AS fi
ON fi.PortfolioCode = p.acct_id
-- Clear down variable values
SET #PortfolioGroup = NULL
--SET #StartDateTime = NULL
SET #EndDateTime = NULL
-- Clear down #DGControl table to allow new record to be inserted
DELETE FROM #DGControl
-- Move to next record
SET #RcdNum = #RcdNum + 1
END -- END WHILE LOOP
Adding the execution of the stored proc usp_Generic_DGControl at the beginning and then clearing it down after each loop stopped the duplication of the records.