How to fix a stored procedure that can recover deleted data - sql-server
I have lately stumbled upon a blog post that talks about a stored procedure called Recover_Deleted_Data_Proc.sql that can apparently recover your deleted data from the .log file.
There is nothing new under the sun, we are going to use fn_dblog.
STEPS TO REPRODUCE
We are first going to create the table:
--Create Table
CREATE TABLE [Test_Table]
(
[Col_image] image,
[Col_text] text,
[Col_uniqueidentifier] uniqueidentifier,
[Col_tinyint] tinyint,
[Col_smallint] smallint,
[Col_int] int,
[Col_smalldatetime] smalldatetime,
[Col_real] real,
[Col_money] money,
[Col_datetime] datetime,
[Col_float] float,
[Col_Int_sql_variant] sql_variant,
[Col_numeric_sql_variant] sql_variant,
[Col_varchar_sql_variant] sql_variant,
[Col_uniqueidentifier_sql_variant] sql_variant,
[Col_Date_sql_variant] sql_variant,
[Col_varbinary_sql_variant] sql_variant,
[Col_ntext] ntext,
[Col_bit] bit,
[Col_decimal] decimal(18,4),
[Col_numeric] numeric(18,4),
[Col_smallmoney] smallmoney,
[Col_bigint] bigint,
[Col_varbinary] varbinary(Max),
[Col_varchar] varchar(Max),
[Col_binary] binary(8),
[Col_char] char,
[Col_timestamp] timestamp,
[Col_nvarchar] nvarchar(Max),
[Col_nchar] nchar,
[Col_xml] xml,
[Col_sysname] sysname
)
And we then insert data into it:
--Insert data into it
INSERT INTO [Test_Table]
([Col_image]
,[Col_text]
,[Col_uniqueidentifier]
,[Col_tinyint]
,[Col_smallint]
,[Col_int]
,[Col_smalldatetime]
,[Col_real]
,[Col_money]
,[Col_datetime]
,[Col_float]
,[Col_Int_sql_variant]
,[Col_numeric_sql_variant]
,[Col_varchar_sql_variant]
,[Col_uniqueidentifier_sql_variant]
,[Col_Date_sql_variant]
,[Col_varbinary_sql_variant]
,[Col_ntext]
,[Col_bit]
,[Col_decimal]
,[Col_numeric]
,[Col_smallmoney]
,[Col_bigint]
,[Col_varbinary]
,[Col_varchar]
,[Col_binary]
,[Col_char]
,[Col_nvarchar]
,[Col_nchar]
,[Col_xml]
,[Col_sysname])
VALUES
(CONVERT(IMAGE,REPLICATE('A',4000))
,REPLICATE('B',8000)
,NEWID()
,10
,20
,3000
,GETDATE()
,4000
,5000
,getdate()+15
,66666.6666
,777777
,88888.8888
,REPLICATE('C',8000)
,newid()
,getdate()+30
,CONVERT(VARBINARY(8000),REPLICATE('D',8000))
,REPLICATE('E',4000)
,1
,99999.9999
,10101.1111
,1100
,123456
,CONVERT(VARBINARY(MAX),REPLICATE('F',8000))
,REPLICATE('G',8000)
,0x4646464
,'H'
,REPLICATE('I',4000)
,'J'
,CONVERT(XML,REPLICATE('K',4000))
,REPLICATE('L',100)
)
GO
We are now going to verify if the data are there:
--Verify the data
SELECT * FROM Test_Table
At this point we need to create the stored procedure. I couldn't paste it here because it's too long but you can download it from the same blog post there is a link to a Box file.
If the query gives you troubles like this:
Msg 50000, Level 16, State 1, Procedure Recover_Deleted_Data_Proc, Line 22 [Batch Start Line 700] The compatibility level should be equal to or greater SQL SERVER 2005 (90)
Msg 50000, Level 16, State 1, Procedure Recover_Deleted_Data_Proc, Line 22 [Batch Start Line 705] The compatibility level should be equal to or greater SQL SERVER 2005 (90)
Is because you have to comment out from line 701 to line 708.
Cool, let's now delete the data from that table:
--Delete the data
DELETE FROM Test_Table
And confirm that the data were deleted:
--Verify the data
SELECT * FROM Test_Table
And here is the last step: we need to try to recover the data using the freshly installed stored procedure.
The author instruct us to use one of these two commands (don't forget to change 'test' with the name of your database):
--Recover the deleted data without date range
EXEC Recover_Deleted_Data_Proc 'test', 'dbo.Test_Table'
or
--Recover the deleted data it with date range
EXEC Recover_Deleted_Data_Proc 'test', 'dbo.Test_Table', '2012-06-01', '2012-06-30'
But the problem is that both returns this error:
(8 rows affected)
(2 rows affected)
(64 rows affected)
(2 rows affected)
(1 row affected)
(1 row affected)
(1 row affected)
(1 row affected)
(1 row affected)
(1 row affected)
Msg 245, Level 16, State 1, Procedure Recover_Deleted_Data_Proc, Line 485 [Batch Start Line 112]
Conversion failed when converting the varchar value '0x41-->01 ; 0001' to data type int.
If I right click on the stored procedure and I click "Modify", I don't see anything particularly fishy at Line 485.
Any idea why this stored procedure is not working?
What is the conversion mentioned?
The code is 10 years old and was written with the assumption that a [PAGE ID] would only ever be expressed as a pair of integers, e.g. 0001:00000138 - however, as you have learned, sometimes that is expressed differently, like 0x41-->01 ; 0001:00000138.
You can fix that problem by adding this inside the cursor:
IF #ConsolidatedPageID LIKE '0x%-->%;%'
BEGIN
SET #ConsolidatedPageID = LTRIM(SUBSTRING(#ConsolidatedPageID,
CHARINDEX(';', #ConsolidatedPageID) + 1, 8000));
END
But then your next problem is when you saved the procedure from the box file it probably changed '†' to some wacky ? character. When I fixed that (using N'†' of course, since Unicode characters should always have N), I still got these error messages:
Msg 537, Level 16, State 3, Procedure Recover_Deleted_Data_Proc, Line 525
Invalid length parameter passed to the LEFT or SUBSTRING function.
Msg 9420, Level 16, State 1, Procedure Recover_Deleted_Data_Proc, Line 651
XML parsing: line 1, character 2, illegal xml character
After 15 minutes of trying to reverse engineer this spaghetti, I gave up. If you need to recover data you deleted, restore a backup. If you don't have a backup, well, that's why we take backups. The fragile scripts people try to create to compensate for not taking backups are exactly why log recovery vendors charge the big bucks.
As an aside, the compatibility level error message is a red herring, totally misleading as the logic is currently written, and completely irrelevant to the problem. But it can be solved if, right before this:
IF ISNULL(#Compatibility_Level,0)<=80
BEGIN
RAISERROR('The compatibility level should ... blah blah',16,1)
RETURN
END
You add this:
IF DB_ID(#Database_Name) IS NULL
BEGIN
RAISERROR(N'Database %s does not exist.',11,1,#Database_name);
RETURN;
END
Or simply not calling those two example calls at the end of the script, since they depend on you having a database called test, which clearly you do not.
Related
Microsoft SQL Server Bulk Insert NOT failing when inserting bigint values into int column
We recently had an identifier column move from int values to bigint values. An ETL process which loads these values was not updated. That process is using SQL bulk insert, and we are seeing incorrect values in the destination table. I would have expected a hard failure. Please note this process has been running successfully for a long time before this issue. Can anyone tell me what the heck SQL Server is doing here!? I know how to fix the situation, but I'm trying to better understand it for the data cleanup effort that I'll need to complete, as well as just for the fact that it looks like black magic! I've been able to re-create this issue in SQL Server 2017 and 2019. Simplified Example CSV file contents: 310067463717 310067463718 310067463719 Example SQL: create table #t (t int) bulk insert #t from 'c:\temp\test.csv' with (datafiletype = 'char', fieldterminator = '|' ) select * from #t Resulting data: 829818405 829818406 829818407 Interestingly, I tried with smaller values, and I do see an error: Example CSV file contents (2147483647 is the largest int value for SQL Server): 310067463717 310067463718 310067463719 2147483647 2147483648 Running the same SQL code as above, I get an error for one row: Msg 4867, Level 16, State 1, Line 4 Bulk load data conversion error (overflow) for row 5, column 1 (t). The resulting data looks like this: 829818405 829818406 829818407 2147483647 I also tried just now with a much higher value, 31006746371945654, and that threw the same overflow error as 2147483648. And last, I did confirm that if I create the table with the column defined as bigint, the data inserted is correct. create table #t (t bigint) bulk insert #t from 'c:\temp\test.csv' with (datafiletype = 'char', fieldterminator = '|' ) select * from #t Resulting data: 2147483647 2147483648 310067463717 310067463718 310067463719
SQL Server :: MASKED WITH (FUNCTION = 'default()'); not working
My goal is to mask columns on SQL Server 2019. I'm following a very easy guide. I run this query against AdventureWorks2014 and I create a copy of Person.Person: SELECT [BusinessEntityID] ,[PersonType] ,[NameStyle] ,[Title] ,[FirstName] ,[MiddleName] ,[LastName] ,[Suffix] ,[EmailPromotion] ,[AdditionalContactInfo] ,[Demographics] ,[rowguid] ,[ModifiedDate] INTO [Person].[PersonMasked] FROM [AdventureWorks2014].[Person].[PersonMasked] ORDER BY BusinessEntityID I'm now going to mask the column FirstName as is written in the guide: ALTER TABLE [AdventureWorks2014].[Person].[PersonMasked] ALTER COLUMN FirstName NVARCHAR(10) MASKED WITH (FUNCTION = 'default()'); And I receive the error: Msg 8152, Level 16, State 30, Line 1 String or binary data would be truncated. The statement has been terminated. Completion time: 2021-11-23T15:32:43.0426983+01:00 Where am I wrong? Where can I find the function FUNCTION = 'default()' in SSMS?
I believe the error is related to the change of nvarchar precision to 10 rather than to the FUNCTION = 'default()' If you have any data with values longer than 10 characters they are going to be truncated and this will result in loss of data. Just do: ALTER TABLE [AdventureWorks2014].[Person].[PersonMasked] ALTER COLUMN FirstName NVARCHAR(50) MASKED WITH (FUNCTION = 'default()'); Helpful links: Altering column size in SQL Server What happens when you modify (reduce) a column's length?
Comparing text to varchar(max) parameter gives String or binary data would be truncated
I can insert the longer than 8k text into a TEXT column, but for some reason I'm not able to do a LIKE with the exact same data. Can someone please explain what I'm doing wrong? BEGIN TRAN CREATE TABLE #log ( body [text] NULL ) GO DECLARE #longString varchar(max) SET #longString = 'Ask Question I am involved in a data migration project. I am getting the following error when I try to insert data from one table into another table (SQL Server 2005): Msg 8152, Level 16, State 13, Line 1 String or binary data would be truncated. The source data columns match the data type and are within the length definitions of the destination table columns so I am at a loss as to what could be causing this error. sql-server tsql sql-server-2005 migration data-migration shareeditflag edited Sep 24 ''18 at 15:31 Lukasz Szozda 79.5k1061105 asked Jun 17 ''11 at 16:24 Jim Evans 3,44092954 Would you mind posting some code, and information about each table? – Kevin Mansel Jun 17 ''11 at 16:27 The tables are both quite large - so I will post only the part of the table definintions that are involved and the code - is that acceptable? – Jim Evans Jun 17 ''11 at 16:30 The table definitions and the code would be great. – IAmTimCorey Jun 17 ''11 at 16:31 add a comment start a bounty 19 Answers active oldest votes 141 You will need to post the table definitions for the source and destination tables for us to figure out where the issue is but the bottom line is that one of your columns in the source table is bigger than your destination columns. It could be that you are changing formats in a way you were not aware of. The database model you are moving from is important in figuring that out as well. shareeditflag edited Sep 4 ''14 at 18:11 Ryan Kohn 7,782104478 answered Jun 17 ''11 at 16:30 IAmTimCorey 13.8k52756 1 Per my comment above - comming shortly :) – Jim Evans Jun 17 ''11 at 16:32 3 I had faced the same problem and had to compare all the column types and sizes of both the tables to fix the issue. – Aziz Shaikh Jun 17 ''11 at 16:40 1 After going thourgh the exeecise of gathering the partial table definitions and then getting my sproc code the offending column jumped out at me like a lightning bolt... Thanks all for your input. – Jim Evans Jun 17 ''11 at 16:47 I can''t tell you how many times I''ve done the same thing. Glad you were able to solve your issue. – IAmTimCorey Jun 17 ''11 at 16:54 I marked you first reply as the answer because it was what led me to find the answer:) – Jim Evans Jun 17 ''11 at 17:25 add a comment | show 1 more comment 0 SQL Server 2019 will finally return more meaningful error message. Binary or string data would be truncated => error message enhancments if you have that error (in production), it''s not obvious to see which column or row this error comes from, and how to locate it exactly. To enable new behavior you need to use DBCC TRACEON(460). New error text from sys.messages: SELECT * FROM sys.messages WHERE message_id = 2628 2628 – String or binary data would be truncated in table ‘%.*ls’, column ‘%.*ls’. Truncated value: ‘%.*ls’. String or Binary data would be truncated: replacing the infamous error 8152 This new message is also backported to SQL Server 2017 CU12 (and in an upcoming SQL Server 2016 SP2 CU), but not by default. You need to enable trace flag 460 to replace message ID 8152 with 2628, either at the session or server level. Note that for now, even in SQL Server 2019 CTP 2.0 the same trace flag 460 needs to be enabled. In a future SQL Server 2019 release, message 2628 will replace message 8152 by default. SQL Server 2017 CU12 also supports this feature. Improvement: Optional replacement for "String or binary data would be truncated" message with extended information in SQL Server 2017 This SQL Server 2017 update introduces an optional message that contains the following additional context information. Msg 2628, Level 16, State 6, Procedure ProcedureName, Line Linenumber String or binary data would be truncated in table ''%.*ls'', column ''%.*ls''. Truncated value: ''%.*ls''. The new message ID is 2628. This message replaces message 8152 in any error output if trace flag 460 is enabled. db<>fiddle demo shareeditflag edited Dec 4 ''18 at 20:20 answered Sep 24 ''18 at 15:29 Lukasz Szozda 79.5k1061105 add a comment 0 I wrote a useful store procedure to help identify and resolve the problem of text truncation (String or binary data would be truncated) when the INSERT SELECT statement is used. It compares fields CHAR, VARCHAR, NCHAR AND NVARCHAR only and returns an evaluation field by field in case of being the possible cause of the error. EXEC dbo.GetFieldStringTruncate SourceTableName, TargetTableName This stored procedure is oriented to the problem of text truncation when an INSERT SELECT statement is made. The operation of this stored procedure depends on the user previously identifying the INSERT statement with the problem. Then inserting the source data into a global temporary table. The SELECT INTO statement is recommended. You must use the same name of the field of the destination table in the alias of each field of the SELECT statement. FUNCTION CODE: DECLARE #strSQL nvarchar(1000) IF NOT EXISTS (SELECT * FROM dbo.sysobjects where id = OBJECT_ID(N''[dbo].[GetFieldStringTruncate]'')) BEGIN SET #strSQL = ''CREATE PROCEDURE [dbo].[GetFieldStringTruncate] AS RETURN'' EXEC sys.sp_executesql #strSQL END GO SET ANSI_NULLS ON GO SET QUOTED_IDENTIFIER ON GO /* ------------------------------------------------------------------------------------------------------------------------ Description: Syntax --------------- dbo.GetFieldStringTruncate(SourceTable, TargetTable) +---------------------------+-----------------------+ | SourceTableName | VARCHAR(255) | +---------------------------+-----------------------+ | TargetTableName | VARCHAR(255) | +---------------------------+-----------------------+ Arguments --------------- SourceTableName The name of the source table. It should be a temporary table using double charp ''##''. E.g. ''##temp'' TargetTableName The name of the target table. It is the table that receives the data used in the INSERT INTO stament. Return Type ---------------- Returns a table with a list of all the fields with the type defined as text and performs an evaluation indicating which field would present the problem of string truncation. Remarks ---------------- This stored procedure is oriented to the problem of text truncation when an INSERT SELECT statement is made. The operation of this stored procedure depends on the user previously identifying the INSERT statement with the problem. Then inserting the source data into a global temporary table. The SELECT INTO statement is recommended. You must use the same name of the field of the destination table in the alias of each field of the SELECT statement. Examples ==================================================================================================== --A. Test basic IF EXISTS (SELECT * FROM sys.objects WHERE OBJECT_ID = OBJECT_ID(N''[dbo].[tblDestino]'') AND TYPE IN (N''U'')) DROP TABLE tblDestino CREATE TABLE tblDestino ( Id INT IDENTITY, Field1 VARCHAR(10), Field2 VARCHAR(12), Field3 VARCHAR(11), Field4 VARCHAR(16), Field5 VARCHAR(5), Field6 VARCHAR(1), Field7 VARCHAR(1), Field8 VARCHAR(6), Field9 VARCHAR(6), Field10 VARCHAR(50), Field11 VARCHAR(50), Field12 VARCHAR(50) ) INSERT INTO dbo.tblDestino ( Field1 , Field2 , Field3 , Field4 , Field5 , ' PRINT LEN(#longString ) INSERT INTO #log ( body) VALUES (#longString ) SELECT * FROM #log WHERE body LIKE #longString DROP TABLE #log ROLLBACK TRAN
As per documentation: pattern Is the specific string of characters to search for in match_expression, and can include the following valid wildcard characters. pattern can be a maximum of 8,000 bytes. source https://learn.microsoft.com/en-us/sql/t-sql/language-elements/like-transact-sql?view=sql-server-2017 so you cannot use string that is longer in 8000 characters after LIKE operator, if you do it will be truncated hence you get an error.
SSMS Bulk inserts = Error + Which line is it?
I'm trying to insert a lot of data with SQL Server Management Studio. This is what I do: I open my file containing a lot of SQL inserts: data.sql I execute it (F5) I get a lot of these: (1 row(s) affected) and some of these: Msg 8152, Level 16, State 13, Line 26 String or binary data would be truncated. The statement has been terminated. Question: How to get the error line number ? Line 26 doesn't seems to be the correct error line number...
This is something that has annoyed SQL Server developers for years. Finally, with SQL Server 2017 CU12 w/ trace flag 460 they give you a better error message, like: Msg 2628, Level 16, State 6, Procedure ProcedureName, Line Linenumber String or binary data would be truncated in table ‘%.*ls’, column ‘%.*ls’. Truncated value: ‘%.*ls A method to get around this now is to add a print statement after each insert. Then, when you see your rows affected print out, you could see what ever you print. ... insert into table1 select... print 'table1 insert complete' insert into table2 select... print 'table2 insert complete' This isn't going to tell you what column, but would narrow it down to the correct insert. You can also add SET NOCOUNT ON at the top of your script if you don't want the rows affected message printed out. Another addition, if you really are using BULK INSERT and weren't just using the term generally, you can specify an ERRORFILE. This will log the row(s) which caused the error(s) in your BULK INSERT command. It's important to know that by default, BULK INSERT will complete if there are 10 errors or less. You can override this by specifying the MAXERRORS in your BULK INSERT command.
Sql Stored Procedures
I'm trying to write a Stored Procedure this is what I have so far Create procedure sp_Create_order #P_nafn varchar(50), #P_fj int, #P_sótt datetime, #F_kt varchar(10), #V_nr int, #L_id int as begin set nocount on if exists( select * from Lotur where L_id=#L_id and #P_sótt between L_hefst and L_pfrest ) INSERT INTO Pantar(P_nafn, P_fj, P_sótt, F_kt, V_nr, L_id) VALUES (#P_nafn, #P_fj, #P_sótt, #F_kt, #V_nr, #L_id) end but I am getting these errors Msg 102, Level 15, State 1, Procedure sp_Create_order, Line 14 Incorrect syntax near ' '. Msg 102, Level 15, State 1, Procedure sp_Create_order, Line 15 Incorrect syntax near ' '. on these lines select * from Lotur where L_id=#L_id and #P_sótt, L_hefst and L_pfrest are all dates and I am tryng to put a condition on saying that nothing should be Inserted unless #P_sótt is equal to or between L_hefst and L_pfrest
Please use meaningful names for your variables Do not create sp for every thing like the one above Modify the query to have SELECT L_ID NOT SELECT * As for the error, probably you have mistyped something
First of all, I wouldn't recommend prefixing your Stored Procedure with sp_. Performance is at least one reason why it's a bad idea. Your stored procedure compiled ok for me. Another recommendation would be to use if exists ( select 1 from Lotur where L_id= #L_id and #P_sótt between L_hefst and L_pfrest ) as opposed to SELECT *. Did you get the error when compiling or trying to use it? EDIT: In response to your post about raising errors, you might want to look at RAISERROR. Here is an example of how to use it in Stored procedures