Hi SO Community I have a Proc that is migrate from tsql into Snowsql . I was doing testing with full table truncate and just loading the data BUT now we are pushing the code into testing phases AND it seems my MERGE has a bug somewhere.
Non-Tech side : A person can have multiple rows of discount of same type on one item. They are caught by [rtrans_lineitm_seq] . So example is you can buy 20 car filters each one shows up as a new line number on the printed receipt . AND if it is a weekly special a discount is applied to each item. That said let just jump into the tech side...
So here is the Snowsql proc code. and dummy record that is causing issue will be posted after the code below.
MERGE INTO DISCOUNT_2 tgt
USING (
select to_varchar(concat(organization_id,rtl_loc_id,to_varchar(replace(to_date(business_date),'-','')),trans_seq,wkstn_id,IFNULL(rtrans_lineitm_seq,0),IFNULL(rtl_price_mod_seq_nbr,0),replace(IFNULL(rtl_price_mod_reasoncode,''),' ',''),replace(IFNULL(discount_code,''),' ',''),replace(IFNULL(deal_id,''),' ','') ) ) as Unique_id
,*
,'PROC_1' AS PROC_NUMBER
,'LOYALTY_2' AS PROC_NAME
,1 as EXECUTION_NUMBER
,current_timestamp(2) as LAST_PROC_EXECUTION_DATE
from (
select 2200 as organization_id
,TH.STORE_NO as rtl_loc_id
,TH.Date as business_date
,TH.TRANSACTION_NO as trans_seq
,case when left(TH.POS_TERMINAL_NO,2) = 'NP' then 1
when left(TH.POS_TERMINAL_NO,2) = 'NS' then 2
when left(TH.POS_TERMINAL_NO,2) = 'NT' then 3 else 0 end as wkstn_id
,TSE.LINE_NO as rtrans_lineitm_seq
,ROW_NUMBER() over (partition by TSE.STORE_NO,TSE.Date,TSE.TRANSACTION_NO,TSE.POS_TERMINAL_NO order by TSE.STORE_NO,TSE.Date,TSE.TRANSACTION_NO,TSE.POS_TERMINAL_NO,TSE.DISCOUNT_AMOUNT ) AS rtl_price_mod_seq_nbr
,CAST(CONCAT(trim(substring(TH.DATE,0,charindex(':',TH.DATE)-4)), ' ' ,substring(TH.Time,charindex(':',TH.Time)-2,length(TH.Time)))AS TIMESTAMP_NTZ(9)) as create_date
,trim(CONCAT('LOYALTY',' ',ifnull(TIE.Information,''))) AS rtl_price_mod_reasoncode
,IFNULL(PERIODIC_DISC_GROUP,TIE.Information) AS discount_code
,IFNULL(abs(TSE.DISCOUNT_AMOUNT),0) AS deal_amt
,null as deal_id
,TH.STAFF_ID as create_user
,null as sales_agt_com
,null as serial_number
from HEADER as TH
join SALES_ENTRY TSE on TSE.TRANSACTION_NO = TH.TRANSACTION_NO and TSE.STORE_NO = TH.STORE_NO and TSE.POS_TERMINAL_NO = TH.POS_TERMINAL_NO
left join CODE_ENTRY TIE on TIE.TRANSACTION_NO = TH.TRANSACTION_NO and TIE.STORE_NO = TH.STORE_NO and TIE.POS_TERMINAL_NO = TH.POS_TERMINAL_NO and TIE.LINE_NO = TSE.LINE_NO
where TH.TRANSACTION_TYPE = 2
and TH.ENTRY_STATUS not in (1,3)
and TIE.TRANSACTION_TYPE = 1
and TIE.INFOCODE = 'LOYALTY'
and TIE.INFORMATION not in ('PPP EXCLUSIVE','PPP Points Discount')
and TH.TRANSACTION_NO >= 20000000
) as a where to_varchar(concat(organization_id,rtl_loc_id,to_varchar(replace(to_date(business_date),'-','')),trans_seq,wkstn_id,IFNULL(rtrans_lineitm_seq,0),IFNULL(rtl_price_mod_seq_nbr,0),replace(IFNULL(rtl_price_mod_reasoncode,''),' ',''),replace(IFNULL(discount_code,''),' ',''),replace(IFNULL(deal_id,''),' ','') ) )
= '2200710320210826200121721126LOYALTYPPPSENIORDISCPPPSENIORDISC'
) AS src
ON (//tgt.Unique_id = src.Unique_id
to_varchar(concat(tgt.organization_id,tgt.rtl_loc_id,to_varchar(replace(to_date(tgt.business_date),'-','')),tgt.trans_seq,tgt.wkstn_id,IFNULL(tgt.rtrans_lineitm_seq,0),IFNULL(tgt.rtl_price_mod_seq_nbr,0),replace(IFNULL(tgt.rtl_price_mod_reasoncode,''),' ',''),replace(IFNULL(tgt.discount_code,''),' ',''),replace(IFNULL(tgt.deal_id,''),' ','') ) ) = src.Unique_id
)
WHEN NOT MATCHED THEN INSERT ( tgt.Unique_id
,tgt.organization_id
,tgt.rtl_loc_id
,tgt.business_date
,tgt.trans_seq
,tgt.wkstn_id
,tgt.rtrans_lineitm_seq
,tgt.rtl_price_mod_seq_nbr
,tgt.create_date
,tgt.rtl_price_mod_reasoncode
,tgt.discount_code
,tgt.deal_amt
,tgt.deal_id
,tgt.create_user
,tgt.sales_agt_com
,tgt.serial_number
,tgt.PROC_NUMBER
,tgt.PROC_NAME
,tgt.EXECUTION_NUMBER
,tgt.LAST_PROC_EXECUTION_DATE
)
values ( src.Unique_id
,src.organization_id
,src.rtl_loc_id
,src.business_date
,src.trans_seq
,src.wkstn_id
,src.rtrans_lineitm_seq
,src.rtl_price_mod_seq_nbr
,src.create_date
,src.rtl_price_mod_reasoncode
,src.discount_code
,src.deal_amt
,src.deal_id
,src.create_user
,src.sales_agt_com
,src.serial_number
,src.PROC_NUMBER
,src.PROC_NAME
,src.EXECUTION_NUMBER
,src.LAST_PROC_EXECUTION_DATE )
WHEN MATCHED THEN UPDATE SET
/* tgt.organization_id = SRC.organization_id
,tgt.rtl_loc_id = SRC.rtl_loc_id
,tgt.business_date = SRC.business_date
,tgt.trans_seq = SRC.trans_seq
,tgt.wkstn_id = SRC.wkstn_id
,tgt.rtrans_lineitm_seq = SRC.rtrans_lineitm_seq
,tgt.rtl_price_mod_seq_nbr = SRC.rtl_price_mod_seq_nbr
,tgt.create_date = SRC.create_date
,tgt.rtl_price_mod_reasoncode = SRC.rtl_price_mod_reasoncode
,tgt.discount_code = SRC.discount_code
,tgt.deal_amt = SRC.deal_amt
,tgt.deal_id = SRC.deal_id
,tgt.create_user = SRC.create_user
,tgt.sales_agt_com = SRC.sales_agt_com
,tgt.serial_number = SRC.serial_number
,tgt.PROC_NUMBER = SRC.PROC_NUMBER
,tgt.PROC_NAME = SRC.PROC_NAME ,*/
tgt.EXECUTION_NUMBER = (SRC.EXECUTION_NUMBER + 1)
,tgt.LAST_PROC_EXECUTION_DATE = current_timestamp(2)
Here is the sample row that I'm testing with
UNIQUE_ID ORGANIZATION_ID RTL_LOC_ID BUSINESS_DATE TRANS_SEQ WKSTN_ID RTRANS_LINEITM_SEQ RTL_PRICE_MOD_SEQ_NBR CREATE_DATE RTL_PRICE_MOD_REASONCODE DISCOUNT_CODE DEAL_AMT DEAL_ID CREATE_USER SALES_AGT_COM SERIAL_NUMBER PROC_NUMBER PROC_NAME EXECUTION_NUMBER LAST_PROC_EXECUTION_DATE
2200710320210826200121721126LOYALTYPPPSENIORDISCPPPSENIORDISC, 2200, 7103, 2021-08-26, 20012172, 1, 1, 26, 29:12.4, LOYALTY PPP SENIOR DISC PPP SENIOR DISC, 0.22, ST7103, 00, PROC_1, LOYALTY_2, 4, 21:53.9,
Error I get
Duplicate row detected during DML action Row Values: ["2200710320210826200121721126LOYALTYPPPSENIORDISCPPPSENIORDISC", 2200, "7103", 18865, 20012172, 1, 1, 26, 1630009752450000000, "LOYALTY PPP SENIOR DISC", "PPP SENIOR DISC", 2200, NULL, "ST7103 00", NULL, NULL, "PROC_1", "LOYALTY_2", 1, 1642853936960000000]
My question is: 9 TIMES OUT OF 10 WHY WOULD THIS GET CAUGHT IN THE "NOT MATCH INSERT" PART on n-th time VS NOT just being moving to the "WHEN MATCHED UPDATE" part???
as you can see in the sample row above. I was able to run the code successfully 4 times BUT WHEN I TRIED TO RUN IT A 5th time it failed with error...
there ARE OTHER CASES I can share if need be.
Any help would be great.
Thanks.
The duplicate exists on the source side and makes it undeterministic.
This behaviour is described in documentation:
Duplicate Join Behavior:
When a merge joins a row in the target table against multiple rows in the source, the following join conditions produce nondeterministic results (i.e. the system is unable to determine the source value to use to update or delete the target row)
In this situation, the outcome of the merge depends on the value specified for the ERROR_ON_NONDETERMINISTIC_MERGE session parameter:
If TRUE (default value), the merge returns an error.
If FALSE, one row from among the duplicates is selected to perform the update or delete; the row selected is not defined.
...
To avoid errors when multiple rows in the data source (i.e. the source table or subquery) match the target table based on the ON condition, use GROUP BY in the source clause to ensure that each target row joins against one row (at most) in the source.
Option number 1:Using session parameter(it is quick fix that will mask the duplicate error but choose source row in undefined manner):
ALTER SESSION SET ERROR_ON_NONDETERMINISTIC_MERGE = FALSE;
Option number 2:
Identify why they are duplicates in the source and change USING part. To find duplicates QUALIFY COUNT(*) OVER(PARTITION BY Unique_id) > 1; is the fastest option:
select to_varchar(concat(organization_id,rtl_loc_id,to_varchar(replace(to_date(business_date),'-','')),trans_seq,wkstn_id,IFNULL(rtrans_lineitm_seq,0),IFNULL(rtl_price_mod_seq_nbr,0),replace(IFNULL(rtl_price_mod_reasoncode,''),' ',''),replace(IFNULL(discount_code,''),' ',''),replace(IFNULL(deal_id,''),' ','') ) ) as Unique_id
,*
,'PROC_1' AS PROC_NUMBER
,'LOYALTY_2' AS PROC_NAME
,1 as EXECUTION_NUMBER
,current_timestamp(2) as LAST_PROC_EXECUTION_DATE
from (
select 2200 as organization_id
,TH.STORE_NO as rtl_loc_id
,TH.Date as business_date
,TH.TRANSACTION_NO as trans_seq
,case when left(TH.POS_TERMINAL_NO,2) = 'NP' then 1
when left(TH.POS_TERMINAL_NO,2) = 'NS' then 2
when left(TH.POS_TERMINAL_NO,2) = 'NT' then 3 else 0 end as wkstn_id
,TSE.LINE_NO as rtrans_lineitm_seq
,ROW_NUMBER() over (partition by TSE.STORE_NO,TSE.Date,TSE.TRANSACTION_NO,TSE.POS_TERMINAL_NO order by TSE.STORE_NO,TSE.Date,TSE.TRANSACTION_NO,TSE.POS_TERMINAL_NO,TSE.DISCOUNT_AMOUNT ) AS rtl_price_mod_seq_nbr
,CAST(CONCAT(trim(substring(TH.DATE,0,charindex(':',TH.DATE)-4)), ' ' ,substring(TH.Time,charindex(':',TH.Time)-2,length(TH.Time)))AS TIMESTAMP_NTZ(9)) as create_date
,trim(CONCAT('LOYALTY',' ',ifnull(TIE.Information,''))) AS rtl_price_mod_reasoncode
,IFNULL(PERIODIC_DISC_GROUP,TIE.Information) AS discount_code
,IFNULL(abs(TSE.DISCOUNT_AMOUNT),0) AS deal_amt
,null as deal_id
,TH.STAFF_ID as create_user
,null as sales_agt_com
,null as serial_number
from HEADER as TH
join SALES_ENTRY TSE on TSE.TRANSACTION_NO = TH.TRANSACTION_NO and TSE.STORE_NO = TH.STORE_NO and TSE.POS_TERMINAL_NO = TH.POS_TERMINAL_NO
left join CODE_ENTRY TIE on TIE.TRANSACTION_NO = TH.TRANSACTION_NO and TIE.STORE_NO = TH.STORE_NO and TIE.POS_TERMINAL_NO = TH.POS_TERMINAL_NO and TIE.LINE_NO = TSE.LINE_NO
where TH.TRANSACTION_TYPE = 2
and TH.ENTRY_STATUS not in (1,3)
and TIE.TRANSACTION_TYPE = 1
and TIE.INFOCODE = 'LOYALTY'
and TIE.INFORMATION not in ('PPP EXCLUSIVE','PPP Points Discount')
and TH.TRANSACTION_NO >= 20000000
) as a where to_varchar(concat(organization_id,rtl_loc_id,to_varchar(replace(to_date(business_date),'-','')),trans_seq,wkstn_id,IFNULL(rtrans_lineitm_seq,0),IFNULL(rtl_price_mod_seq_nbr,0),replace(IFNULL(rtl_price_mod_reasoncode,''),' ',''),replace(IFNULL(discount_code,''),' ',''),replace(IFNULL(deal_id,''),' ','') ) )
= '2200710320210826200121721126LOYALTYPPPSENIORDISCPPPSENIORDISC'
QUALIFY COUNT(*) OVER(PARTITION BY Unique_id) > 1;
If the query returns more than one row it means source query is not producing unique_id and requires redesign.
I have been at this for some time and am frustrated. Below is the complete SQL statement I am trying to execute. Simply put, create a temp table from a UNION Select(s) statements.
I can run the SELECTs with the UNION just fine. I can run the separate SELECTS just fine. I can run separate INSERT INTOs just fine. The moment I add an INSERT INTO SELECT * FROM (...) at the top and close it at the bottom I get a syntax error:
Msg 102, Level 15, State 1, Line 75
Incorrect syntax near ')'.
I was hoping this would be simple, but I tried everything I know to get this to work, and cannot. is there something, perhaps with the UNION that creates the syntax error?
INSERT INTO #Tmp6 SELECT * FROM
(
SELECT
[ItemStatus]
,[Item_Number]
,[Item_Name]
,[Default_Sales_Name]
,[Special_Order_Item]
,[Returnable]
,[Return_Number_Of_Days]
,[Is_Drop_Ship]
,[Is_JIT]
,[Vendor_Name]
,[Vendor_Number]
,[Default_Sales_Price]
,[Cost_Of_Goods]
,[Purchase_Cost]
,[Additional_Cost1_Text]
,[Additional_Cost1_Value]
,[Additional_Cost2_Text]
,[Additional_Cost2_Value]
,[Additional_Cost3_Text]
,[Additional_Cost3_Value]
,[UPC_Value]
,[Weight]
,[DimWeight]
,[Discontinued_Date]
,[Personalization_Template_Number]
,[Harmonized_Code]
,[Base_Category]
,[Sub_Category]
,[End_Category]
,[RowID]
FROM [dbo].[RAIProductsStg] p
WHERE ItemStatus <> 'Discontinued' OR
p.Item_Number IN (SELECT i.Item_Number FROM RAIInventoryStg i WHERE p.Item_Number = i.Item_Number AND Quantity > 0)
UNION
SELECT
[ItemStatus]
,[Item_Number]
,[Item_Name]
,[Default_Sales_Name]
,[Special_Order_Item]
,[Returnable]
,[Return_Number_Of_Days]
,[Is_Drop_Ship]
,[Is_JIT]
,[Vendor_Name]
,[Vendor_Number]
,[Default_Sales_Price]
,[Cost_Of_Goods]
,[Purchase_Cost]
,[Additional_Cost1_Text]
,[Additional_Cost1_Value]
,[Additional_Cost2_Text]
,[Additional_Cost2_Value]
,[Additional_Cost3_Text]
,[Additional_Cost3_Value]
,[UPC_Value]
,[Weight]
,[DimWeight]
,[Discontinued_Date]
,[Personalization_Template_Number]
,[Harmonized_Code]
,[Base_Category]
,[Sub_Category]
,[End_Category]
,[RowID]
FROM [dbo].[RAIProductsStg] p
LEFT JOIN RAIParentChildStg pc ON p.Item_Number = pc.ParentItemNumber
WHERE(LEN(p.Item_Number) = 6 AND p.ItemStatus = 'Discontinued') AND
(pc.ChildItemNumber IN
(SELECT i.Item_Number FROM RAIInventoryStg i
WHERE pc.ChildItemNumber = i.Item_Number AND i.Quantity > 0))
)
Please try this. Put alias after last bracket.
INSERT INTO #Tmp6
SELECT * FROM
(
SELECT
[ItemStatus]
,[Item_Number]
,[Item_Name]
,[Default_Sales_Name]
,[Special_Order_Item]
,[Returnable]
,[Return_Number_Of_Days]
,[Is_Drop_Ship]
,[Is_JIT]
,[Vendor_Name]
,[Vendor_Number]
,[Default_Sales_Price]
,[Cost_Of_Goods]
,[Purchase_Cost]
,[Additional_Cost1_Text]
,[Additional_Cost1_Value]
,[Additional_Cost2_Text]
,[Additional_Cost2_Value]
,[Additional_Cost3_Text]
,[Additional_Cost3_Value]
,[UPC_Value]
,[Weight]
,[DimWeight]
,[Discontinued_Date]
,[Personalization_Template_Number]
,[Harmonized_Code]
,[Base_Category]
,[Sub_Category]
,[End_Category]
,[RowID]
FROM [dbo].[RAIProductsStg] p
WHERE ItemStatus <> 'Discontinued' OR
p.Item_Number IN (SELECT i.Item_Number FROM RAIInventoryStg i WHERE p.Item_Number = i.Item_Number AND Quantity > 0)
UNION
SELECT
[ItemStatus]
,[Item_Number]
,[Item_Name]
,[Default_Sales_Name]
,[Special_Order_Item]
,[Returnable]
,[Return_Number_Of_Days]
,[Is_Drop_Ship]
,[Is_JIT]
,[Vendor_Name]
,[Vendor_Number]
,[Default_Sales_Price]
,[Cost_Of_Goods]
,[Purchase_Cost]
,[Additional_Cost1_Text]
,[Additional_Cost1_Value]
,[Additional_Cost2_Text]
,[Additional_Cost2_Value]
,[Additional_Cost3_Text]
,[Additional_Cost3_Value]
,[UPC_Value]
,[Weight]
,[DimWeight]
,[Discontinued_Date]
,[Personalization_Template_Number]
,[Harmonized_Code]
,[Base_Category]
,[Sub_Category]
,[End_Category]
,[RowID]
FROM [dbo].[RAIProductsStg] p
LEFT JOIN RAIParentChildStg pc ON p.Item_Number = pc.ParentItemNumber
WHERE(LEN(p.Item_Number) = 6 AND p.ItemStatus = 'Discontinued') AND
(pc.ChildItemNumber IN
(SELECT i.Item_Number FROM RAIInventoryStg i
WHERE pc.ChildItemNumber = i.Item_Number AND i.Quantity > 0))
)c
Place the bracket after the second select statement. That should do it.
INSERT INTO TableA
(
SELECT A, B, C
FROM TableB
UNION
SELECT D, E, F
FROM TableC
)
I'm trying to INNER JOIN three tables and get it to return values for XML file, but the values that are from second or third table are returned as
Msg 207, Level 16, State 1, Procedure PROC_Generate_XML_AdForm, Line
18 Invalid column name 'VPrSKzNazev'.
Msg 207, Level 16, State 1, Procedure PROC_Generate_XML_AdForm, Line
20 Invalid column name 'VPrURLCZ1'.
Msg 207, Level 16, State 1, Procedure PROC_Generate_XML_AdForm, Line
22 Invalid column name 'VPrSlevaCZ1'.
Here is the code:
SELECT #xmlVar = (
SELECT t1.IDS AS 'product_id'
,t1.VPrSKzNazev AS 'product_name'
,t1.VPrPodkolekce AS 'product_category_id'
,'http://www.foo.com/' + t1.VPrURLCZ1 AS 'product_deeplink'
,'http://www.foo.com/media-photo/' + t1.IDS + '/370/370.jpg' AS 'product_image'
,CAST(ROUND((100 - t1.ProdejDPH) / 100 * t1.VPrSlevaCZ1, 0) AS INT) AS 'product_price'
FROM SKz AS t1
INNER JOIN VTbProdDalsi
ON t1.IDS = VTbProdDalsi.VPrSKzIDS
INNER JOIN VTbProdDalsi2
ON t1.IDS = VTbProdDalsi2.VPrSKzIDS
WHERE t1.VPrIsMain = 1
AND (
(
SELECT SUM(SKz.StavZ - SKz.ObjedP)
FROM SKz
WHERE IDS = t1.IDS
) > 0
OR (
SELECT SUM(ISNULL(SKz.VPrDodPocet, 0) - SKz.ObjedP)
FROM SKz
WHERE IDS = t1.IDS
) > 0
)
FOR XML PATH('product')
,ROOT('products')
)
I get the idea that it's probably answered somewhere already, but since I'm new to SQL, I'm not sure how to adapt the answers.
I'm on Microsoft SQL Server Management Studio (SQL Server 2008 R2).
Thanks for any replies.
It seems like your problem is residing right here:
,t1.VPrSKzNazev AS 'product_name'
,t1.VPrPodkolekce AS 'product_category_id'
,'http://www.foo.com/' + t1.VPrURLCZ1 AS 'product_deeplink'
I would make sure those names match up with what is in table Skz.
Also, if you can't figure it out, can you give us a list of columns from Skz?
Solved.
The solution was not to use the joined table t1 for variables that were not in the SKz table.
select #xmlVar =
(
select
t1.IDS as 'product_id',
VTbProdDalsi.VPrSKzNazev as 'product_name',
t1.VPrPodkolekce as 'product_category_id',
'http://www.ringit.cz/' + VTbProdDalsi2.VPrURLCZ1 as 'product_deeplink',
'http://www.ringit.cz/media-photo/' + t1.IDS + '/370/370.jpg' as 'product_image',
CAST(ROUND(t1.ProdejDPH - t1.ProdejDPH * VTbProdDalsi.VPrSlevaCZ1/100, 0) AS int) as 'product_price'
from SKz as t1
inner join VTbProdDalsi on t1.IDS = VTbProdDalsi.VPrSKzIDS
inner join VTbProdDalsi2 on t1.IDS = VTbProdDalsi2.VPrSKzIDS
where t1.VPrIsMain = 1 AND t1.VPrGledis = 1 AND
((SELECT SUM(SKz.StavZ - SKz.ObjedP) FROM SKz WHERE IDS = t1.IDS) > 0 OR (SELECT SUM(ISNULL(SKz.VPrDodPocet, 0) - SKz.ObjedP) FROM SKz WHERE IDS = t1.IDS) > 0)
FOR XML PATH ('product'), ROOT ('products')
)