I have this table:
CREATE TABLE Vendors
(
VendorID NUMERIC(10) NOT NULL,
VendorName CHAR(50) NOT NULL,
VendorAddress VARCHAR(30) NULL,
VendorCityName VARCHAR(20) NOT NULL,
VendorStateName CHAR(2) NOT NULL,
VendorZip VARCHAR(10) NULL,
VendorContactName CHAR(50) NOT NULL,
VendorContactPhone VARCHAR(12) NOT NULL,
VendorContactEmail VARCHAR(20) NOT NULL,
VendorSpecialty CHAR(20) NOT NULL
CONSTRAINT VendorsPK PRIMARY KEY (VendorID)
);
And this insert:
INSERT INTO Vendors(VendorID, VendorName, VendorAddress,
VendorCityName, VendorStateName, VendorZip, VendorContactName,
VendorContactPhone, VendorContactEmail, VendorSpecialty)
VALUES(151330, 'Hyperion', '77 West 66th Street', 'New York',
'NY', 10023, 'John Hinks', '212-337-6564',
'jhinks#hyperionbooks.com', 'Popular fiction')
Why does this statement yield the 8152 error?
VendorContactEmail is only 20 bytes. Your e-mail address on the first line (jhinks#hyperionbooks.com) is longer than that - 24 bytes. And many e-mail addresses will be longer. Who decided to only allow 20 characters in the e-mail address column? According to the standard, this should be VARCHAR(320) - 64 characters for <localpart> + 1 for # + 255 for <domain>.
As for the error message itself, finding the culprit is easier today than it was back then.
String or Binary data would be truncated: replacing the infamous error 8152
How to Force an Operation that Results in Truncation to Execute Anyway
Do you know what truncation is?
Do you know what is going to be truncated?
Is it your intention to truncate long data to fit?
If and and only if you answer YES to the above questions, you can force your inserts to ignore the warning and run anyway. If you answered no to any of the above, read on.
SET ANSI_WARNINGS OFF;
-- Your operation TSQL here.
SET ANSI_WARNINGS ON;
(source)
What is Truncation in this Context
Truncation is simply putting as much of a value as will fit into the column and then discarding any portion that doesn't fit. For example, truncating The quick brown fox jumps over the lazy dog to 13 characters would be The quick bro
Why would I receive this error message
You receive this error when you attempt to insert data that won't fit into a destination column definition because the data is too short.
I'm trying to insert many rows of data and want to determine which rows don't fit
Aaron's excellent answer notes that if you are running SQL Server 2019 or newer, you'll actually get a message that contains the column and value that won't fit - which is awesome! But if you aren't running a version that new, read on for tips.
If you receive this error message while attempting to bulk insert many rows of data, you might try splitting the insert into multiple inserts and running them separately to narrow down where the long value is.
Alternatively, you could insert the data into a new temp table and search said temp table for values that won't fit into your destination table.
--(insert into new temp table #Vendors)
INSERT INTO #Vendors(VendorID, VendorName, VendorAddress,
VendorCityName, VendorStateName, VendorZip, VendorContactName,
VendorContactPhone, VendorContactEmail, VendorSpecialty)
VALUES(151330, 'Hyperion', '77 West 66th Street', 'New York',
'NY', 10023, 'John Hinks', '212-337-6564',
'jhinks#hyperionbooks.com', 'Popular fiction')
Then query for rows that don't fit.
--(query for values that don't fit)
SELECT *,
LEN(VendorContactEmail) AS Length
FROM #Vendors
WHERE LEN(VendorContactEmail) > 20 --set your destination column length is here
See also LEN and DATALENGTH documentation for information on whitespace handling and binary data lengths.
You can also minimize the length of column values by using following expression:
LEFT(columnName, 250) + '...'
I tried and it works.
Related
A strange issue that occurs sometimes.
The environment is Microsoft SQL Server Web (64-bit) on Windows Server 2016 Standard (10.0). SQL Server version is 14.0.1000.169
I have a table, TableA with a column ColEncrypted and primary key column PK. The data type of PK is int and it is auto incremented. The datatype of ColEncrypted is varbinary(500) and the column is nullable. This column is used to stored date value in encrypted format. Before encrypting the date, it is first converted to nvarchar in format yyyyMMdd. So for e.g. April 1, 2020 becomes '20200401'. Thus the data to be encrypted is always 8 characters in length. This value is then encrypted using the T-SQL function EncryptByPassPhrase. So the encrypting is done as follows.
First a row is inserted in the TableA without any value in ColEncrypted. So ColEncrypted is NULL at insert time
An update statement is called to update the ColEncrypted column with the value to be encrypted as follows:
UPDATE TableA SET ColEncrypted = ENCRYPTBYPASSPHRASE(#passphrase,
#date,1,CONVERT(VARBINARY,PK))
WHERE PK = #Id
#passphrase is a nvarchar(50) value of length 12
#date is nvarchar(25) with date value in format yyyyMMdd. For e.g.
April 1, 2020 is '20200401'
PK is the primary key of the TableA and is an identity column of
data type int. It is used as authenticator. Thus each row has a
unique authenticator.
Decryption is done in a stored procedure as follows:
SELECT CONVERT(NVARCHAR,DECRYPTBYPASSPHRASE(#passphrase,ColEncrypted,1,CONVERT(VARBINARY,PK))) FROM
TableA WHERE PK = #Id
Now sometimes the decrypted value is NULL. Strangely the same value when encrypted again is decyrpted properly.
The one interesting thing I found is that the datalength of encrypted value for which the decrypted value is NULL is 67 whereas the datalength of encrypted value for which decryption occurs properly is 68, if this is something to consider.
Edit 1: (In response to Andrew's comments)
According to MS, VARBINARY when used in cast without specifying a size defaults to 30, which is sufficient enough. Look at the Remarks section in this article. https://learn.microsoft.com/en-us/sql/t-sql/data-types/binary-and-varbinary-transact-sql?view=sql-server-ver15
So that can't be a issue otherwise it would have failed each time. As for sysname, if you look at the example provided here https://learn.microsoft.com/en-us/sql/t-sql/functions/encryptbypassphrase-transact-sql?view=sql-server-ver15
, CreditCardId is used as authenticator by converting it to varbinary where a sysname datatype is expected. CreditCardId is a varchar datatype but nowhere I have found that int datatype cannot be used as long as I am converting it to varbinary.
Again as I mentioned this issue does not occur consistently. i.e. for the same input values for which the decryption fails, if I re-run the code for encryption, the decryption succeeds the next time. Therefore I suspect the issue is in the SQL Server engine. I have raised this question in SQL Product feedback but so far there is no response from them
I'm working on SQL Server 2008.
I delete all data from a table and then I try to insert value to the table. Here's the code:
TRUNCATE TABLE [dbo].[STRAT_tmp_StratMain]
INSERT INTO [dbo].[STRAT_tmp_StratMain] ([FileNum])
SELECT [dbo].[STRAT_tmp_Customer].[NumericFileNumber]
FROM [dbo].[STRAT_tmp_Customer];
The FileNum in STRAT_tmp_StratMain is float number and is also index and can't be null.
NumericFileNumber is float and can be null but is never null and there are no duplicates in it (each row is unique number).
The table STRAT_tmp_StratMain contain much more fields but all can be null and also has a defualt values.
When I try to run this query I get the error:
Msg 8152, Level 16, State 4, Line 1 String or binary data would be
truncated. The statement has been terminated.
I tried also to do simply:
INSERT INTO [dbo].[STRAT_tmp_StratMain] ([FileNum]) Values (1);
Still get the same error.
Any ideas?
Thanks,
Ilan
I am not able to reproduce your issue. When I run this code on SQL Server 2008, I get no error:
DECLARE #tt TABLE (FileNum float NOT NULL);
INSERT INTO #tt (FileNum) VALUES (1);
Check the Default constraints on all the columns in your target table and make sure none of them would try to insert a string value that would truncated by the datatype limitations of the column.
example: SomeColumn varchar(1) DEFAULT 'Hello'
This due to the data you are trying to insert does not fit in the field: if you have a defined length of (say) 10 or 50 characters but the data you are trying to insert is longer than that.
For some reason, the SSIS Lookup transformation seems to be checking the cache for a NCHAR(128) value instead of a NVARCHAR(128) value. This results in a whole bunch of appended whitespace on the value being looked up and causes the lookup to fail to find a match.
On the Lookup transformation, I configured it to have No Cache so that it always goes to the database so I could trace with SQL Profiler and see what it was looking up. This is what it captured (notice the whitespace ending at the single quote on the second last line - requires horizontal scrolling):
exec sp_executesql N'
select *
from (
SELECT SurrogateKey, NaturalKey, SomeInt
FROM Dim_SomeDimensionTable
) [refTable]
where [refTable].[NaturalKey] = #P1
and [refTable].[SomeInt] = #P2'
,N'#P1 nchar(128)
,#P2 smallint'
,N'VALUE '
,8
Here's the destination table's schema:
CREATE TABLE [dbo].[dim_SomeDimensionTable] (
[SurrogateKey] [int] IDENTITY(1,1) NOT NULL,
[NaturalKey] [nvarchar](128) NOT NULL,
[SomeInt] [smallint] NOT NULL
)
What I am trying to figure out is why SSIS is checking the NaturalKey value as NCHAR(128) and how I can get it to perform the lookup as NVARCHAR(128) without the whitespace.
Things I've tried:
I have LTRIM() and RTRIM() on the SQL Server source query.
Before the Lookup, I have used a Derived Column transformation to add a new column with the original value TRIM()'d (this trimmed column is the one I'm passing to the Lookup transformation).
Before and after the Lookup, I multicasted the rows and sent them to a unicode Flat File Destination and there was no white space in either case.
Before the lookup, I looked at the metadata on the data flow path and it shows the value as having data type DT_WSTR with length 128.
Any ideas would be greatly appreciated!
It doesn't make any difference.
You need to look elsewhere for the source of your problem (perhaps the column has a case sensitive collation for example).
Trailing white space is only significant to SQL Server in LIKE comparisons, not = comparisons as documented here.
SQL Server follows the ANSI/ISO SQL-92 specification (Section 8.2,
, General rules #3) on how to compare strings
with spaces. The ANSI standard requires padding for the character
strings used in comparisons so that their lengths match before
comparing them. The padding directly affects the semantics of WHERE
and HAVING clause predicates and other Transact-SQL string
comparisons. For example, Transact-SQL considers the strings 'abc' and
'abc ' to be equivalent for most comparison operations.
The only exception to this rule is the LIKE predicate...
You can also easily see this by running the below.
USE tempdb;
CREATE TABLE [dbo].[Dim_SomeDimensionTable] (
[SurrogateKey] [int] IDENTITY(1,1) NOT NULL,
[NaturalKey] [nvarchar](128) NOT NULL,
[SomeInt] [smallint] NOT NULL
)
INSERT INTO [dbo].[Dim_SomeDimensionTable] VALUES ('VALUE',8)
exec sp_executesql N'
select *
from (
SELECT SurrogateKey, NaturalKey, SomeInt
FROM Dim_SomeDimensionTable
) [refTable]
where [refTable].[NaturalKey] = #P1
and [refTable].[SomeInt] = #P2'
,N'#P1 nchar(128)
,#P2 smallint'
,N'VALUE '
,8
Which returns the single row
I have a SQL table in which some columns, when viewed in SQL Server Manager, contain <Unable to read data>. Does anyone know how to query for <Unable to read data>? I can individually modify the data in this column with update table set column = NULL where key = 'value', but how can I find whether additional rows exist with this bad data?
I would recommend against replacing the data. There is nothing wrong with it, is just that SSMs cannot display it properly in the Edit panel. The data in the database itself is perfectly fine, from your description.
This script shows the problem:
create table test (id int not null identity(1,1) primary key,
large_value numeric(38,0));
go
insert into test (large_value) values (1);
insert into test (large_value) values (12345678901234567890123456789012345678);
insert into test (large_value) values (1234567890123456789012345678901234567);
insert into test (large_value) values (123456789012345678901234567890123456);
insert into test (large_value) values (12345678901234567890123456789012345);
insert into test (large_value) values (1234567890123456789012345678901234);
insert into test (large_value) values (123456789012345678901234567890123);
insert into test (large_value) values (12345678901234567890123456789012);
insert into test (large_value) values (1234567890123456789012345678901);
insert into test (large_value) values (123456789012345678901234567890);
insert into test (large_value) values (12345678901234567890123456789);
insert into test (large_value) values (NULL);
go
select * from test;
go
The SELECT will work fine, but showing the Edit Top 200 Rows in object explorer will not:
There is a Connect Item for this issue. SSMS 2012 still exhibits the same problem.
If we look at the Numeric and Decimal details we'll see that the problem occurs at a weird boundary, at precision 29 which is actually not a SQL Server boundary (precision 28 is):
Precision Storage bytes
1 - 9 5
10-19 9
20-28 13
29-38 17
If we check the .Net (SSMS is a managed application) decimal precision table we can see quickly where the crux of the issue is: Precision is 28-29 significant digits. So the .Net decimal type cannot map high precision (>29) SQL Server numeric/decimal types.
This will affect not only SSMS display, but your applications as well. Specialized applications like SSIS will use high precisions representation like DT_NUMERIC:
DT_NUMERIC An exact numeric value with a fixed precision and scale.
This data type is a 16-byte unsigned integer with a separate sign, a
scale of 0 - 38, and a maximum precision of 38.
Now back to your problem: you can discover invalid entries by simply looking at the value. Knowing that the C# representation range can accommodate values between approximate (-7.9 x 1028 to 7.9 x 1028) / (100 to 28)` (the range depends on the scale) you can search for values outside the range on each column (the actual values to search between will depend on the column scale). But that begs the question 'what to replace the data with?'.
I would recommend instead using dedicated tools for import export, tools that are capable of handling high precision numeric values. SSIS is the obvious candidate. But even the modest bcp.exe would also fit the bill.
BTW if your values are actually incorrect (ie. true corruption) then I would recommend running DBCC CHECKTABLE (...) WITH DATA_PURITY:
DATA_PURITY
Causes DBCC CHECKDB to check the database for column values that are not valid or out-of-range. For example, DBCC CHECKDB detects
columns with date and time values that are larger than or less than
the acceptable range for the datetime data type; or decimal or
approximate-numeric data type columns with scale or precision values
that are not valid.
For databases created in SQL Server 2005 and later, column-value integrity checks are enabled by default and do not require the
DATA_PURITY option. For databases upgraded from earlier versions of
SQL Server, column-value checks are not enabled by default until DBCC
CHECKDB WITH DATA_PURITY has been run error free on the database.
After this, DBCC CHECKDB checks column-value integrity by default.
Q: How can this issue arise for a datetime column?
use tempdb;
go
create table test(d datetime)
insert into test (d) values (getdate())
select %%physloc%%, * from test;
-- Row is on page 0x9100000001000000
dbcc traceon(3604,-1);
dbcc page(2,1,145,3);
Memory Dump #0x000000003FA1A060
0000000000000000: 10000c00 75f9ff00 6aa00000 010000 ....uùÿ.j .....
Slot 0 Column 1 Offset 0x4 Length 8 Length (physical) 8
dbcc writepage(2,1,145, 100, 8, 0xFFFFFFFFFFFFFFFF)
dbcc checktable('test') with data_purity;
Msg 2570, Level 16, State 3, Line 2 Page (1:145), slot 0 in object ID
837578022, index ID 0, partition ID 2882303763115671552, alloc unit ID
2882303763120062464 (type "In-row data"). Column "d" value is out of
range for data type "datetime". Update column to a legal value.
As suggested above ,these errors usually occurs when Precision and scale are not preserved .If your comfortable with SSIS then you can achieve to get those rows which are corrupt .Taking the values which Martin Smith created
CREATE TABLE T(ID int ,C DECIMAL(38,0));
INSERT INTO T VALUES(1,9999999999999999999999999999999999999)
The above table reproduces the error . Here the first column represents the primary key . I inserted around 1000 rows out of which few were corrupted values . Below is the SSIS package design
In the Data Conversion ,i took the column C which had errors and tried to cast it to Decimal(38,0) .Since a conversion or truncation error will occur ,therefore i redirected the error rows to an OLEDB command which basically updates the table and sets the column to NULL
Update T
Set C=NULL
where ID=?
The value of C and ID will be directed to oledb command .In case if there is no error then i'm just inserting into a table ( Actually no need to do this ).This will work if you have a primary key column in your table .
In case if there is any error in date time column a sql query can be written to verify the format of datetime values .Please go through the MSDN link for valid date time value
Select * from YourTable where ISDATE(Col)!=1
I think you can fetch data with cursor. please try again with cursor query such as below query :
DECLARE VerifyCursor CURSOR FOR
SELECT *
FROM MyTable
WHILE 1=1 BEGIN
BEGIN Try
FETCH FIRST FROM VerifyCursor INTO #Column1, #Column2, ...
INSERT INTO #MyTable2(Column1, Column2,...)
VALUES (#Column1, #Column2, ...)
END TRY
BEGIN CATCH
END CATCH
IF (##FETCH_STATUS<>0) BREAK
End
OPEN VerifyCursor
CLOSE VerifyCursor
DEALLOCATE VerifyCursor
Replacing the bad data is simple with an update:
UPDATE table SET column = NULL WHERE key_column = 'Some value'
My table :
log_id bigint
old_value xml
new_value xml
module varchar(50)
reference_id bigint
[transaction] varchar(100)
transaction_status varchar(10)
stack_trace ntext
modified_on datetime
modified_by bigint
Insert Query :
INSERT INTO [dbo].[audit_log]
([old_value],[new_value],[module],[reference_id],[transaction]
,[transaction_status],[stack_trace],[modified_on],[modified_by])
VALUES
('asdf','asdf','Subscriber',4,'_transaction',
'_transaction_status','_stack_trace',getdate(),555)
Error :
Msg 8152, Level 16, State 14, Line 1
String or binary data would be truncated.
The statement has been terminated.
Why is that ???
You're trying to write more data than a specific column can store. Check the sizes of the data you're trying to insert against the sizes of each of the fields.
In this case transaction_status is a varchar(10) and you're trying to store 19 characters to it.
this type of error generally occurs when you have to put characters or values more than that you have specified in Database table like in this case:
you specify
transaction_status varchar(10)
but you actually trying to store
_transaction_status
which contain 19 characters.
that's why you faced this type of error in this code..
This error is usually encountered when inserting a record in a table where one of the columns is a VARCHAR or CHAR data type and the length of the value being inserted is longer than the length of the column.
I am not satisfied how Microsoft decided to inform with this "dry" response message, without any point of where to look for the answer.