Catch Block not Handling the error - sql-server

Question

I am updating and inserting the bulk records from web api(in form of data table) in my table using the Merge statement. The insert and update are working fine but when I am sending a corrupt value.Catch block is not handling that value. Here is my code :
//Creating type to fetch the data table from web api(have to go with this approach only)
CREATE TYPE [dbo].[CustomerType] AS TABLE(
[Id] [int] NULL,
[Name] [nvarchar](100) NULL,
[Country] [nvarchar](50) NULL,
[Date] [datetime] NULL
)
// Stored proc for update and insert using merge
CREATE PROCEDURE Update_Customers
#tblCustomers CustomerType READONLY
AS
BEGIN
BEGIN TRY
MERGE INTO Customers c1
USING #tblCustomers c2
ON c1.CustomerId=c2.Id
WHEN MATCHED THEN
UPDATE SET c1.Name = c2.Name
,c1.Country = c2.Country,
c1.date =c2.date
WHEN NOT MATCHED THEN
INSERT VALUES(c2.Id, c2.Name,c2.date, c2.Country);
END TRY
BEGIN CATCH
//My table for logging the error
INSERT INTO ERROR_LOG(ERROR_LINE,ERROR_MESSAGE,PROC_NAME)
VALUES (ERROR_LINE(),ERROR_MESSAGE(),ERROR_PROCEDURE)
END CATCH
END
Thanks in advance

Answer 1

The problem is SQL does not treat errors the way you think it does.
SQL Server maintains an ACID state.
Atomic: All or nothing. All work is broken into transactional statements, which must be successful or the entire modification/creation is reverted, or rollback, to the previous working state. You can be explicit what transaction a work is in by using BEGIN TRANSACTION/COMMIT TRANSACTION or using ROLLBACK TRANSACTION. Read More: Transaction Statements and ROLLBACK TRANSACTION
Consistent: Every transaction must leave SQL Server in a state that is valid. For example, while DROP TABLE MyTable; may be a valid transaction, if MyTable has dependencies (i.e. Foreign Key Constraints), then SQL Server would be left in an inconsistent state and will rollback the transaction to the last consistent state.
Isolated: Every transaction occurs in its own time and space. We say Serialized to be specific. Isolation allows for multiple, even similar statements to be made at one time on the Server and process every one of them exclusively from each other. The term Blocking refers to when a statement is waiting for a transaction to commit and a Dead Block is when both transactions are waiting for each other indefinitely.
Durability: Unlike a software program that lives in memory and can be lost by a sudden loss of power, SQL Server's transactions are permanent to the disk once committed. This provides finality to a statement. This is also where the LOG file comes in place, since it records what transactions are performed on the database.
READ MORE: ACID PROPERTIES
I mention all of this since your BEGIN TRY/CATCH block looks for these issues.
Treat your tables as sets of data
Recall SQL is based on the relational set theory. SQL is best when it can perform actions on groups/sets of data/objects. It is completely incompatible with inheritance, and using cursive logic is possible, but inefficient at best.
So in your ETL process, treat the data as a whole set.
Instead, by treating your data as a whole set, you can transform the minor misspellings/errors of the user/web interface, and isolate actual errors of datatypes that you wish to catalogue by using the predicate clause (WHERE/HAVING/ON)
An example of what you could do is similar to the following:
CREATE TABLE #ETLTable(ID INT NOT NULL
, Name VARCHAR(50) NOT NULL
, Comment VARCHAR(50) NULL
, Country VARCHAR(50) NOT NULL
, Dated VARCHAR(50)); --implicitly declared as allowing NULL values unless the type denies it
CREATE TABLE #MyTable (ID INT NOT NULL
, Name VARCHAR(50) NOT NULL
, Comment VARCHAR(50) NULL
, Country VARCHAR(50) NOT NULL
, Dated DATE);
CREATE TABLE #ERROR_TABLE (ObjectID INT NULL
, encrypted INT
, text VARCHAR(MAX)
, start_time DATETIME2
, Table_ID INT
, Table_Name VARCHAR(50)
, Table_Country VARCHAR(50)
, Table_Dated VARCHAR(20));
CREATE TABLE #ACTIONS ( [Action] VARCHAR(50)
, [inserted_ID] int
, inserted_Name VARCHAR(50)
, inserted_Country VARCHAR(50)
, inserted_Date Date
, deleted_ID int
, deleted_Name VARCHAR(50)
, deleted_Country VARCHAR(50)
, deleted_Date Date)
INSERT INTO #MyTable (ID, Name, Country, Dated)
VALUES (1, 'Mary', 'USA', '12/23/12')
, (2, 'Julio', 'Mexico', '12/25/12')
, (3, 'Marx', 'USA', '11/11/12')
, (4, 'Ann', 'USA', '11/27/12');
INSERT INTO #ETLTable(ID, Name, Country, Comment, Dated)
VALUES (1,'Mary', 'USA', 'Valid Date', '12/23/12')
, (2,'Julio', 'Mexico', 'Invalid Date', '12-25,12')
, (3,'Marx', 'USA', 'Valid but incorrect Date', '12-11/25') --this actually means YY-MM-DD
, (4,'Ann','USA', 'Not Matching Date', '12-23-12')
, (5, 'Hillary', 'USA', 'New Entry', '11/24/12');
/*SQL Server is fairly flexible to datatypes entries, so be explicit.
Note the date highlighted. This will fail your code since it is of datatype DATETIME. CAST is implicit anyways, and should not be depended on in important queries. Theoretically, you could catch this with your TRY/CATCH block, but the entire Merge statement would be rolled back...an expensive and probably unacceptable cost.
You should proof your INSERTIONS of errors before you start expensive transactions like the MERGE statement. In this example, I knew what dates were being entered and that only the Japanese version might be entered. Dates are very finicky (DATE has no format) and best handled outside the merge statement altogether. */
;WITH CTE AS (
SELECT ID, Name, Country, Comment, ISNULL(TRY_CAST(Dated AS Date), CAST(CONVERT(datetime, Dated, 11) AS DATE) ) AS Dated --TRY_CAST returns a NULL if it cannot succeed and will not fail your query.
FROM #ETLTable
WHERE ISDATE(Dated) = 1
)
MERGE INTO #MyTable TGT
USING CTE SRC ON TGT.ID = SRC.ID
AND TGT.Name = SRC.Name
WHEN MATCHED AND SRC.Dated > TGT.Dated
THEN UPDATE SET TGT.Dated = SRC.Dated
, TGT.Comment = SRC.Comment
WHEN NOT MATCHED BY TARGET
THEN INSERT(ID, Name, Country, Comment, Dated) VALUES (SRC.ID, SRC.Name, SRC.Country, SRC.Comment, SRC.DATED)
OUTPUT $action AS [Action]
, inserted.ID
, inserted.Name
, inserted.Country
, inserted.Dated
, deleted.ID
, deleted.Name
, deleted.Country
, deleted.Dated
INTO #Actions;
/* Note, you would have to run this query separately, as it only records active transactions. */
--CREATE PROC MyProject
--AS BEGIN
--WAITFOR DELAY '00:00:10'
--Print 'ME'
--END
;WITH CTE AS (
SELECT t.objectid, encrypted, text, start_time
FROM sys.dm_exec_requests AS r
CROSS APPLY sys.dm_exec_sql_text(r.sql_handle) AS t
INNER JOIN (SELECT object_id FROM sys.procedures
WHERE object_id = OBJECT_ID('MyProject') ) B ON t.objectid = B.object_id )
INSERT INTO #ERROR_TABLE (ObjectID, encrypted, text, start_time, Table_ID, Table_Name, Table_Country, Table_Dated)
SELECT CTE.objectid, CTE.encrypted, CTE.text, CTE.start_time, B.Table_ID, B.Table_Name, B.Table_Country, B.Table_Dated
FROM CTE
RIGHT OUTER JOIN ( SELECT ID AS Table_ID
, Name AS Table_Name
, Country AS Table_Country
, Dated AS Table_Dated
FROM #ETLTable
WHERE ISDATE(Dated) = 0) B ON 1 = 1
SELECT * FROM #ERROR_TABLE
SELECT * FROM #Actions
SELECT * FROM #MyTable
Note that the script does not break anything in the SQL Server and in fact separates the bad inputs BEFORE the Merge statement.
Notice the useful information in the #ERROR_TABLE. You can actually utilize and make recommendations from this information live. This is your goal in error catching.
As a reminder, understand your RAISERROR or TRY/CATCH will only work on entire transactions...a unit of work. If you need to gracefully allow a Merge statement to fail, fine. But understand that proper ETL will guarantee such expensive statements never fail.
So please, no matter what, perform all your ETL processes BEFORE you write to your final tables. This is the point of staging tables...so you can filter out illegal or misspellings in your bulk tables.
To do anything less is to be lazy and unprofessional. Learn the right habits and they will help save you from future disaster.