I'm need to run a calculation for month every day. If the month period, exists already, I need to update it, else I need to create a new row for the new month.
Currently, I've written
declare #period varchar(4) = '0218'
DECLARE #Timestamp date = GetDate()
IF EXISTS(select * from #output where period=#period)
/* UPDATE #output SET --- same calculation as below ---*/
ELSE
SELECT
#period AS period,
SUM(timecard.tworkdol) AS dol_local,
SUM(timecard.tworkdol/currates.cdrate) AS dol_USD,
SUM(timecard.tworkhrs) AS hrs,
#Timestamp AS timestamp
FROM dbo.timecard AS timecard
INNER JOIN dbo.timekeep ON timecard.ttk = timekeep.tkinit
INNER JOIN dbo.matter with (nolock) on timecard.tmatter = matter.mmatter
LEFT JOIN dbo.currates with (nolock) on matter.mcurrency = currates.curcode
AND currates.trtype = 'A'
AND timecard.tworkdt BETWEEN currates.cddate1
AND currates.cddate2
WHERE timekeep.tkloc IN('06','07') AND
timecard.twoper = #period
SELECT * FROM #output;
How can simply update my row with the new data from my select.
Not sure what RDBMS are you using, but in SQL Server something like this would update the #output table with the results of the SELECT that you placed in the ELSE part:
UPDATE o
SET o.dol_local = SUM(timecard.tworkdol),
SET o.dol_USD = SUM(timecard.tworkdol/currates.cdrate),
SET o.hrs = SUM(timecard.tworkhrs),
set o.timestamp = #Timestamp
FROM #output o
INNER JOIN dbo.timecard AS timecard ON o.period = timecard.twoper
INNER JOIN dbo.timekeep ON timecard.ttk = timekeep.tkinit
INNER JOIN dbo.matter with (nolock) on timecard.tmatter = matter.mmatter
LEFT JOIN dbo.currates with (nolock) on matter.mcurrency = currates.curcode
AND currates.trtype = 'A'
AND timecard.tworkdt BETWEEN currates.cddate1
AND currates.cddate2
WHERE timekeep.tkloc IN('06','07') AND
timecard.twoper = #period
Also, I think you want to do an INSERT in the ELSE part, but you are doing just a SELECT, so I guess you should fix that too
The answer to this will vary by SQL dialect, but the two main approaches are:
1. Upsert (if your DBMS supports it), for example using a MERGE statement in SQL Server.
2. Base your SQL on an IF:
IF NOT EXISTS (criteria for dupes)
INSERT INTO (logic to insert)
ELSE
UPDATE (logic to update)
Related
I am updating data in while loop (in stored procedure) which is killing the performance badly. Is there any alternate (best) solution with relatively better performance while updating table.
I have read about CTE but not able to transform in my current problem
WHILE (#Counter <= (SELECT COUNT(id) FROM #TopicsIds))
BEGIN
SELECT #CurrentTopicId = TopicId
FROM #TopicsIds
WHERE id = #Counter;
UPDATE #Modules
SET TopicId = #CurrentTopicId
WHERE id = #Counter;
SET #Counter = #Counter + 1;
END;
UPDATE m
SET TopicId = t.TopicId
FROM #Modules m
JOIN #TopicsIds t ON m.Id = t.Id
Why to use loop for update, use direct update query like
update a set a.TopicId = b.TopicId
from #Modules as a
inner join #TopicsIds as b on a.id = b.id
There are many answers on here about how triggers should work but I wonder is mine unique in that the code runs on insert however the #type variable gets a false answer. If I run the query in isolation with the table reference I get a correct answer returned; it is only as part of the trigger that I get a false.
Part of the query calls a function but again this all works when done in isolation so I am wondering is there something else in my trigger causing it to fail that I would be unaware of?
SQL:
ALTER TRIGGER [dbo].[MPH_I_TEST_GROUP_REQUEST_PRINTS]
ON [dbo].[MPH_PRINT_TEST_QUEUE]
FOR INSERT,UPDATE
AS
DECLARE #V_Refno NUMERIC(10)
DECLARE #V_User_Modif VARCHAR(30)
BEGIN
SET #V_User_Modif = suser_sname()
SELECT #V_Refno = tgreq_refno FROM inserted
--> Get details of test type
DECLARE #car_sponts VARCHAR(8000)
DECLARE #type VARCHAR(50)
SET #type = 'blank'
SELECT #car_sponts = CONVERT(VARCHAR(8000),sypro.long_value)
FROM
system_profiles sypro (NOLOCK)
WHERE
sypro.code = 'MPH_CARD_PRT_SPONTS'
AND (sypro.archv_flag = 'N' OR sypro.archv_flag IS NULL)
--> Update type based on tests ordered
SELECT
#type = CASE WHEN COUNT(*) > 0 THEN 'CARDIO' ELSE 'type fail' END --> Does the request belong in a service point noted in system profile MPH_CARD_PRT_SPONTS
FROM
test_group_requests tgreq (NOLOCK)
INNER JOIN
test_form_requests tfreq (NOLOCK) ON tgreq.tgreq_refno = tfreq.tgreq_refno
INNER JOIN
test_requests tereq (NOLOCK) ON tfreq.tfreq_refno = tereq.tfreq_refno
INNER JOIN
test_definitions tstdf (NOLOCK) ON tereq.tstdf_refno = tstdf.tstdf_refno
INNER JOIN
test_locations tstlc (NOLOCK) ON tstlc.tstdf_refno = tstdf.tstdf_refno
WHERE
tgreq.tgreq_refno IN (SELECT tgreq_refno FROM inserted)
AND (tstlc.archv_flag = 'N' OR tstlc.archv_flag IS NULL)
AND tstlc.spont_refno IN
(
SELECT DISTINCT spont.spont_refno FROM service_points spont (NOLOCK) WHERE (spont.archv_flag = 'N' OR spont.archv_flag IS NULL) AND spont.code IN (SELECT Item FROM [dbo].[MPH_PARSELIST](#car_sponts))
)
--> Update test type to print queue
UPDATE mph_print_test_queue
SET mph_print_test_queue.type = 'updated ' + #type
FROM
inserted,mph_print_test_queue
WHERE
mph_print_test_queue.tgreq_refno = inserted.tgreq_refno
END
So currently all records get a type of 'updated type fail' even when they meet the criteria for 'cardio' to be inserted/updated.
Thanks in advance.
Phil.
Tried this update also:
UPDATE mph_print_test_queue
SET mph_print_test_queue.type = 'updated cardio'
FROM
inserted,test_group_requests tgreq (NOLOCK)
INNER JOIN
test_form_requests tfreq (NOLOCK) ON tgreq.tgreq_refno = tfreq.tgreq_refno
INNER JOIN
test_requests tereq (NOLOCK) ON tfreq.tfreq_refno = tereq.tfreq_refno
INNER JOIN
test_definitions tstdf (NOLOCK) ON tereq.tstdf_refno = tstdf.tstdf_refno
INNER JOIN
test_locations tstlc (NOLOCK) ON tstlc.tstdf_refno = tstdf.tstdf_refno
WHERE
tgreq.tgreq_refno = inserted.tgreq_refno
AND (tstlc.archv_flag = 'N' OR tstlc.archv_flag IS NULL)
AND tstlc.spont_refno IN
(
SELECT DISTINCT spont.spont_refno FROM service_points spont (NOLOCK) WHERE (spont.archv_flag = 'N' OR spont.archv_flag IS NULL) AND spont.code IN (SELECT Item FROM [dbo].[MPH_PARSELIST](#car_sponts))
)
I finally got the light bulb moment on my way home. Don't link the update to inserted table but just batch update according to matching records.
Sorry for not getting this before posting. Long day.
I have the following trigger is causing me grief.
ALTER TRIGGER [dbo].[T_DMS_Factory_I]
ON [dbo].[jobrun]
AFTER INSERT
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for trigger here
IF NOT EXISTS (select * from inserted i
inner join [db2].[dbo].[table2] t2 on t2.JobRunID = i.jobrun_id)
UPDATE [db2].[dbo].[table2]
SET ProdDt = i.jobrun_proddt,
CreateDt = GETDATE(),
JobrunID = i.jobrun_id,
Start = i.jobrun_time,
End = i.jobrun_stachgtm,
Status = i.jobrun_status,
ActiveDuration = i.jobrun_duration,
TotalDuration = i.jobrun_duration
FROM inserted i
INNER JOIN jobmst jm ON jm.jobmst_id = i.jobmst_id
WHERE jm.jobmst_alias = 'blah'
END
The above works fine but it updates ALL the rows with the same data which I don't want. I only want it to update the row where the time is after before the time of the insert from another column called baseEnd
WHERE jm.jobmst_alias = 'blah' AND baseEnd <= i.jobrun_time AND JobRunID IS NOT NULL
That is what I'd imagine should work but it isn't. What am I doing wrong?
Looking at the tables in your code
UPDATE [db2].[dbo].[table2]
SET
....
FROM inserted i
inner join jobmst jm on jm.jobmst_id = i.jobmst_id
where jm.jobmst_alias = 'blah'
shouldn't you have table2 in the FROM Clause if you're going to update matching records in it.
Otherwise you're just getting some records back from the inserted and jobmst tables and firing values from one of them into all the records in table2
That's why it's not working anyway
It's difficult to see quite what the logic you're trying to implement is but this might be what you need
UPDATE t2
SET ProdDt = i.jobrun_proddt,
CreateDt = GETDATE(),
JobrunID = i.jobrun_id,
Start = i.jobrun_time,
End = i.jobrun_stachgtm,
Status = i.jobrun_status,
ActiveDuration = i.jobrun_duration,
TotalDuration = i.jobrun_duration
FROM inserted i
inner join jobmst jm on jm.jobmst_id = i.jobmst_id
inner join [db2].[dbo].[table2] t2 ON t2.baseEnd <= i.jobrun_time
where jm.jobmst_alias = 'blah'
Our DBA has changed a function to be a procedure, so I am amending some of my procedures to cater for this. I've come across a problem with this in one of my procedures where I have a while loop.
I populate my temp table from the new procedure (#DGContol), and then have the following while loop:
SELECT #MinRcd = MIN(RcdNum) FROM #PortfolioDisclosure
SELECT #MaxRcd = MAX(RcdNum) FROM #PortfolioDisclosure
SET #RcdNum = #MinRcd
WHILE #RcdNum <= #MaxRcd
BEGIN
-- Temporarily assign values to variables
SELECT
#PortfolioGroup = PortfolioCode
, #EndDateTime = MaxPositionDate_DetailedDisclosure
FROM #PortfolioDisclosure
WHERE RcdNum = #RcdNum
INSERT INTO #PositionsTable
SELECT
fi.ID_ISIN AS [Fund_ISIN]
, RTRIM(a.acct_id) AS [Internal_Portfolio_Code]
, a.acct_desc AS [Portfolio/Fund_Name]
, CONVERT(CHAR(11),p.as_of_tms,103) AS [Portfolio_Date]
, a.alt_curr_cde AS [Portfolio_Base_Ccy]
, i.iss_desc AS [Security_Description]
, RTRIM(i.pref_iss_id) AS [Security_ID SEDOL/Internal]
, RTRIM(ia.iss_id) AS [Security_ID ISIN]
, i.denom_curr_cde AS [Denomination_Ccy]
, SUM(p.valval_alt_cmb_amt) OVER (PARTITION BY RTRIM(a.acct_id))
AS [Total_Fund_Value]
, p.orig_quantity AS [Shares/Par_Value]
, p.valval_alt_cmb_amt AS [Market_Value]
, p.fld5_rte AS [Pct_of_NAV]
, SUM(CASE WHEN i.issue_cls1_cde = '010' THEN p.valval_alt_cmb_amt ELSE 0 END) OVER (PARTITION BY a.acct_id)
AS [Cash/Cash_Equivalents]
, i.inc_proj_cmb_rte AS [Coupon_Rate]
, CONVERT(CHAR(11),i.mat_exp_dte,103) AS [Maturity_Date]
FROM dw_position_dg AS p
INNER JOIN #DGControl AS dgc -- [M]onthly, [M]ost recent position
ON dgc.DataGrpCtlNum = p.data_grp_ctl_num
INNER JOIN dw_ivw_acct AS a WITH (NOLOCK)
ON a.acct_id = p.acct_id
INNER JOIN dw_issue_dg AS i WITH (NOLOCK)
ON i.instr_id = p.instr_id
LEFT OUTER JOIN dw_issue_alt_id AS ia WITH (NOLOCK)
ON ia.instr_id = i.instr_id
AND ia.id_ctxt_typ = 'ISIN'
INNER JOIN #PortfolioDisclosure AS fi
ON fi.PortfolioCode = p.acct_id
and fi.MaxPositionDate_DetailedDisclosure >= p.as_of_tms
WHERE RTRIM(a.acct_id) NOT IN ( SELECT xref.internal_value FROM dbo.DP_CrossReference as xref
WHERE xref.Codeset_type_id = 10401
AND xref.Originator_id = 'DataVendorPortfolioExclusion')
-- Clear down variable values
SET #PortfolioGroup = NULL
--SET #StartDateTime = NULL
SET #EndDateTime = NULL
-- Move to next record
SET #RcdNum = #RcdNum + 1
END -- END WHILE LOOP
However this returns lots of duplicate records. If I replace the temp table #DGControl with the original function then I get the correct number of records.
I don't really know what the issue would be or how I could re code this while loop so that using the table #DGControl I get the correct number of records. Can anyone help?
Have you compared the output of SELECT * FROM OldFunction(args..) with EXEC NewStoredProcedure args...? If so, does the data returned look the same or has the duplication crept in when the DBA refactored the function to a proc.
If so you may need to de dupe the output from the sp first then run it through your remaining code. In short, if you are using the same arguments for both but they give you different results then you'll need to go back to the DBA.
You aren't using either of your local variables (#PortfolioGroup, #EndDateTime) in this code. It is just inserting the same record set N number of times. Also, I think you can write this as a single select query without using temp tables or while loops and it will make it less confusing.
Thank you for your feedback. i worked out the issue. My code looks like the following now:
BEGIN
SELECT #MinRcd = MIN(RcdNum) FROM #PortfolioDisclosure
SELECT #MaxRcd = MAX(RcdNum) FROM #PortfolioDisclosure
SET #RcdNum = #MinRcd
WHILE #RcdNum <= #MaxRcd
BEGIN
-- Temporarily assign values to variables
SELECT
#PortfolioGroup = PortfolioCode
, #EndDateTime = MaxPositionDate_DetailedDisclosure
FROM #PortfolioDisclosure
WHERE RcdNum = #RcdNum
-- Insert DGControl record into table based on the MaxPositionDate_DetailedDisclosure from Portfolio Disclosure function
INSERT INTO #DGControl
EXEC InfoPortal..usp_Generic_DGControl '', '', #PortfolioGroup, '1', #EndDateTime, #EndDateTime, #PeriodType, 'M', 'POSITION' -- [M]onthly, [M]ost recent position
-- Insert into #PositionsTable
INSERT INTO #PositionsTable
SELECT
fi.ID_ISIN AS [Fund_ISIN]
, RTRIM(a.acct_id) AS [Internal_Portfolio_Code]
, a.acct_desc AS [Portfolio/Fund_Name]
, CONVERT(CHAR(11),p.as_of_tms,103) AS [Portfolio_Date]
, a.alt_curr_cde AS [Portfolio_Base_Ccy]
, i.iss_desc AS [Security_Description]
, RTRIM(i.pref_iss_id) AS [Security_ID SEDOL/Internal]
, RTRIM(ia.iss_id) AS [Security_ID ISIN]
, i.denom_curr_cde AS [Denomination_Ccy]
, SUM(p.valval_alt_cmb_amt) OVER (PARTITION BY RTRIM(a.acct_id))
AS [Total_Fund_Value]
, p.orig_quantity AS [Shares/Par_Value]
, p.valval_alt_cmb_amt AS [Market_Value]
, p.fld5_rte AS [Pct_of_NAV]
, SUM(CASE WHEN i.issue_cls1_cde = '010' THEN p.valval_alt_cmb_amt ELSE 0 END) OVER (PARTITION BY a.acct_id)
AS [Cash/Cash_Equivalents]
, i.inc_proj_cmb_rte AS [Coupon_Rate]
, CONVERT(CHAR(11),i.mat_exp_dte,103) AS [Maturity_Date]
FROM dw_position_dg AS p
INNER JOIN #DGControl AS dgc
ON dgc.DataGrpCtlNum = p.data_grp_ctl_num
INNER JOIN dw_ivw_acct AS a WITH (NOLOCK)
ON a.acct_id = p.acct_id
INNER JOIN dw_issue_dg AS i WITH (NOLOCK)
ON i.instr_id = p.instr_id
LEFT OUTER JOIN dw_issue_alt_id AS ia WITH (NOLOCK)
ON ia.instr_id = i.instr_id
AND ia.id_ctxt_typ = 'ISIN'
INNER JOIN #PortfolioDisclosure AS fi
ON fi.PortfolioCode = p.acct_id
-- Clear down variable values
SET #PortfolioGroup = NULL
--SET #StartDateTime = NULL
SET #EndDateTime = NULL
-- Clear down #DGControl table to allow new record to be inserted
DELETE FROM #DGControl
-- Move to next record
SET #RcdNum = #RcdNum + 1
END -- END WHILE LOOP
Adding the execution of the stored proc usp_Generic_DGControl at the beginning and then clearing it down after each loop stopped the duplication of the records.
MyTableA has several million records. On regular occasions every row in MyTableA needs to be updated with values from TheirTableA.
Unfortunately I have no control over TheirTableA and there is no field to indicate if anything in TheirTableA has changed so I either just update everything or I update based on comparing every field which could be different (not really feasible as this is a long and wide table).
Unfortunately the transaction log is ballooning doing a straight update so I wanted to chunk it by using UPDATE TOP, however, as I understand it I need some field to determine if the records in MyTableA have been updated yet or not otherwise I'll end up in an infinite loop:
declare #again as bit;
set #again = 1;
while #again = 1
begin
update top (10000) MyTableA
set my.A1 = their.A1, my.A2 = their.A2, my.A3 = their.A3
from MyTableA my
join TheirTableA their on my.Id = their.Id
if ##ROWCOUNT > 0
set #again = 1
else
set #again = 0
end
is the only way this will work if I add in a
where my.A1 <> their.A1 and my.A2 <> their.A2 and my.A3 <> their.A3
this seems like it will be horribly inefficient with many columns to compare
I'm sure I'm missing an obvious alternative?
Assuming both tables are the same structure, you can get a resultset of rows that are different using
SELECT * into #different_rows from MyTable EXCEPT select * from TheirTable and then update from that using whatever key fields are available.
Well, the first, and simplest solution, would obviously be if you could change the schema to include a timestamp for last update - and then only update the rows with a timestamp newer than your last change.
But if that is not possible, another way to go could be to use the HashBytes function, perhaps by concatenating the fields into an xml that you then compare. The caveat here is an 8kb limit (https://connect.microsoft.com/SQLServer/feedback/details/273429/hashbytes-function-should-support-large-data-types) EDIT: Once again, I have stolen code, this time from:
http://sqlblogcasts.com/blogs/tonyrogerson/archive/2009/10/21/detecting-changed-rows-in-a-trigger-using-hashbytes-and-without-eventdata-and-or-s.aspx
His example is:
select batch_id
from (
select distinct batch_id, hash_combined = hashbytes( 'sha1', combined )
from ( select batch_id,
combined =( select batch_id, batch_name, some_parm, some_parm2
from deleted c -- need old values
where c.batch_id = d.batch_id
for xml path( '' ) )
from deleted d
union all
select batch_id,
combined =( select batch_id, batch_name, some_parm, some_parm2
from some_base_table c -- need current values (could use inserted here)
where c.batch_id = d.batch_id
for xml path( '' ) )
from deleted d
) as r
) as c
group by batch_id
having count(*) > 1
A last resort (and my original suggestion) is to try Binary_Checksum? As noted in the comment, this does open the risk for a rather high collision rate.
http://msdn.microsoft.com/en-us/library/ms173784.aspx
I have stolen the following example from lessthandot.com - link to the full SQL (and other cool functions) is below.
--Data Mismatch
SELECT 'Data Mismatch', t1.au_id
FROM( SELECT BINARY_CHECKSUM(*) AS CheckSum1 ,au_id FROM pubs..authors) t1
JOIN(SELECT BINARY_CHECKSUM(*) AS CheckSum2,au_id FROM tempdb..authors2) t2 ON t1.au_id =t2.au_id
WHERE CheckSum1 <> CheckSum2
Example taken from http://wiki.lessthandot.com/index.php/Ten_SQL_Server_Functions_That_You_Have_Ignored_Until_Now
I don't know if this is better than adding where my.A1 <> their.A1 and my.A2 <> their.A2 and my.A3 <> their.A3, but I would definitely give it a try (assuming SQL Server 2005+):
declare #again as bit;
set #again = 1;
declare #idlist table (Id int);
while #again = 1
begin
update top (10000) MyTableA
set my.A1 = their.A1, my.A2 = their.A2, my.A3 = their.A3
output inserted.Id into #idlist (Id)
from MyTableA my
join TheirTableA their on my.Id = their.Id
left join #idlist i on my.Id = i.Id
where i.Id is null
/* alternatively (instead of left join + where):
where not exists (select * from #idlist where Id = my.Id) */
if ##ROWCOUNT > 0
set #again = 1
else
set #again = 0
end
That is, declare a table variable for collecting the IDs of the rows being updated and use that table for looking up (and omitting) IDs that have already been updated.
A slight variation on the method would be to use a local temporary table instead of a table variable. That way you would be able to create an index on the ID lookup table, which might result in better performance.
If schema change is not possible. How about using trigger to save off the Ids that have changed. And only import/export those rows.
Or use trigger to export it immediately.