Originally i thought that this issue is related to C# TransactionScope or Dapper.NET. But since i have tested the sql in SSMS and the issue remains i assume that it's a pure sql issue.
This is the (simplified) update which should update a varchar(40) column. I don't get any errors and row-count is 1. The old SparePartDescription is EC801/¦USB/Exch Acc/JP/PE bag:
declare #rowCount int
UPDATE [tabSparePart] SET
[SparePartDescription] = 'EC801/╡USB/Exch Acc/JP/PE bag'
WHERE ([idSparePart] = 13912)
set #rowCount = ##ROWCOUNT
select #rowCount
So the only difference are these special characters: ╡ and ¦.
Maybe you have an idea why i cannot update this column.
Well, the special character you are using:
╡
Is not supported in ASCII. See:
DECLARE #x VARCHAR(32) = 'EC801/╡USB/Exch Acc/JP/PE bag';
SELECT #x;
Result:
EC801/¦USB/Exch Acc/JP/PE bag
So the update is working fine, technically, it's just not doing what you want, because in order to fit into the ASCII space, it has to substitute your character for one that is valid.
In order to support that character, you'll have to use Unicode for your column (and maybe a specific collation, I'm not sure). This works fine:
DECLARE #x NVARCHAR(32) = N'EC801/╡USB/Exch Acc/JP/PE bag';
SELECT #x;
Result:
EC801/╡USB/Exch Acc/JP/PE bag
It will be important to specify the N prefix on string literals that contain such characters...
Related
When I SELECT from the table, the data that I stored is stored as question marks.
#word is a parameter in my stored procedure, and the value comes from the C# code:
string word = this.Request.Form["word"].ToString();
cmd.Parameters.Add("#word", System.Data.SqlDbType.NVarChar).Value = word;
My stored procedure is like this:
CREATE PROCEDURE ....
(
#word nvarchar(500)
...
)
Insert into rub_translate (language_id,name)
values (8 ,#word COLLATE HEBREW_CI_AS )
My database, and the column, is using the SQL_Latin1_General_CP1_CI_AS collation and I cannot change them.
Can anybody give me a solution how can I solve this problem just by modifying the column or the table?
In order for this to work you need to do the following:
Declare the input parameter in the app code as NVARCHAR (you have done this)
Declare the input parameter in the stored procedure as NVARCHAR (no code is shown for this)
Insert or Update a column in a table that is defined as NVARCHAR (you claim that this is the case)
When using NVARCHAR it does not matter what the default Collation of the Database is. And actually, when using NVARCHAR, it won't matter what the Collation of the column in the table is, at least not for properly inserting the characters.
Also, specifying the COLLATE keyword in the INSERT statement is unnecessary and wouldn't help anyway. If you have the stored procedure input parameter defined as VARCHAR, then the characters are already converted to ? upon coming into the stored procedure. And if the column is actually defined as VARCHAR (you haven't provided the table's DDL) then if the Collation isn't Hebrew_* then there is nothing you can do (besides change either the datatype to NVARCHAR or the Collation to a Hebrew_ one).
If those three items listed at the top are definitely in place, then the last thing to check is the input value itself. Since this is a web app, it is possible that the encoding of the page itself is not set correctly. You need to set a break point just at the cmd.Parameters.Add("#word", System.Data.SqlDbType.NVarChar).Value = word; line and confirm that the value held in the word variable contains Hebrew characters instead of ?s.
ALSO: you should never create a string parameter without specifying the max length/size. The default is 30 (in this case, sometimes it's 1), yet the parameter in the stored procedure is defined as NVARCHAR(500). This could result in silent truncation ("silent" meaning that it will not cause an error, it will just truncate the value). Instead, you should always specify the size. For example:
cmd.Parameters.Add("#word", System.Data.SqlDbType.NVarChar, 500).Value = word;
You could just insert it as-is, since it's unicode and then select it with a proper collation:
declare #test table([name] nvarchar(500) collate Latin1_General_CI_AS);
declare #word nvarchar(500) = N'זה טקסט.';
insert into #test ( [name] ) values ( #word );
select [t].[name] collate Hebrew_CI_AS from #test as [t]
Or you can change the collation of that one column in the table all together. But remember that there is a drawback of having a different collation from your database in one or more columns: you will need to add the collate statement to queries when you need to compare data between different collations.
The year is 2010.
SQL Server licenses are not cheap.
And yet, this error still does not indicate the row or the column or the value that produced the problem. Hell, it can't even tell you whether it was "string" or "binary" data.
Am I missing something?
A quick-and-dirty way of fixing these is to select the rows into a new physical table like so:
SELECT * INTO dbo.MyNewTable FROM <the rest of the offending query goes here>
...and then compare the schema of this table to the schema of the table into which the INSERT was previously going - and look for the larger column(s).
I realize that this is an old one. Here's a small piece of code that I use that helps.
What this does, is returns a table of the max lengths in the table you're trying to select from. You can then compare the field lengths to the max returned for each column and figure out which ones are causing the issue. Then it's just a simple query to clean up the data or exclude it.
DECLARE #col NVARCHAR(50)
DECLARE #sql NVARCHAR(MAX);
CREATE TABLE ##temp (colname nvarchar(50), maxVal int)
DECLARE oloop CURSOR FOR
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'SOURCETABLENAME' AND TABLE_SCHEMA='dbo'
OPEN oLoop
FETCH NEXT FROM oloop INTO #col;
WHILE (##FETCH_STATUS = 0)
BEGIN
SET #sql = '
DECLARE #val INT;
SELECT #val = MAX(LEN(' + #col + ')) FROM dbo.SOURCETABLENAME;
INSERT INTO ##temp
( colname, maxVal )
VALUES ( N''' + #col + ''', -- colname - nvarchar(50)
#val -- maxVal - int
)';
EXEC(#sql);
FETCH NEXT FROM oloop INTO #col;
END
CLOSE oloop;
DEALLOCATE oloop
SELECT * FROM ##temp
DROP TABLE ##temp;
Another way here is to use binary search.
Comment half of the columns in your code and try again. If the error persists, comment out another half of that half and try again. You will narrow down your search to just two columns in the end.
You could check the length of each inserted value with an if condition, and if the value needs more width than the current column width, truncate the value and throw a custom error.
That should work if you just need to identify which is the field causing the problem. I don't know if there's any better way to do this though.
Recommend you vote for the enhancement request on Microsoft's site. It's been active for 6 years now so who knows if Microsoft will ever do anything about it, but at least you can be a squeaky wheel: Microsoft Connect
For string truncation, I came up with the following solution to find the max lengths of all of the columns:
1) Select all of the data to a temporary table (supply column names where needed), e.g.
SELECT col1
,col2
,col3_4 = col3 + '-' + col4
INTO #temp;
2) Run the following SQL Statement in the same connection (adjust the temporary table name if needed):
DECLARE #table VARCHAR(MAX) = '#temp'; -- change this to your temp table name
DECLARE #select VARCHAR(MAX) = '';
DECLARE #prefix VARCHAR(256) = 'MAX(LEN(';
DECLARE #suffix VARCHAR(256) = ')) AS max_';
DECLARE #nl CHAR(2) = CHAR(13) + CHAR(10);
SELECT #select = #select + #prefix + name + #suffix + name + #nl + ','
FROM tempdb.sys.columns
WHERE object_id = object_id('tempdb..' + #table);
SELECT #select = 'SELECT ' + #select + '0' + #nl + 'FROM ' + #table
EXEC(#select);
It will return a result set with the column names prefixed with 'max_' and show the max length of each column.
Once you identify the faulty column you can run other select statements to find extra long rows and adjust your code/data as needed.
I can't think of a good way really.
I once spent a lot of time debugging a very informative "Division by zero" message.
Usually you comment out various pieces of output code to find the one causing problems.
Then you take this piece you found and make it return a value that indicates there's a problem instead of the actual value (in your case, should be replacing the string output with the len(of the output)). Then manually compare to the lenght of the column you're inserting it into.
from the line number in the error message, you should be able to identify the insert query that is causing the error. modify that into a select query to include AND LEN(your_expression_or_column_here) > CONSTANT_COL_INT_LEN for the string various columns in your query. look at the output and it will give your the bad rows.
Technically, there isn't a row to point to because SQL didn't write the data to the table. I typically just capture the trace, run it Query Analyzer (unless the problem is already obvious from the trace, which it may be in this case), and quickly debug from there with the ages old "modify my UPDATE to a SELECT" method. Doesn't it really just break down to one of two things:
a) Your column definition is wrong, and the width needs to be changed
b) Your column definition is right, and the app needs to be more defensive
?
The best thing that worked for me was to put the rows first into a temporary table using select .... into #temptable
Then I took the max length of each column in that temp table. eg. select max(len(jobid)) as Jobid, ....
and then compared that to the source table field definition.
I'm storing encrypted passwords in the database, It worked perfect so far on MachineA. Now that I moved to MachineB it seems like the results gets corrupted in the table.
For example: ù9qÆæ\2 Ý-³Å¼]ó will change to ?9q??\2 ?-³?¼]? in the table.
That's the query I use:
ALTER PROC [Employees].[pRegister](#UserName NVARCHAR(50),#Password VARCHAR(150))
AS
BEGIN
DECLARE #Id UNIQUEIDENTIFIER
SET #Id = NEWID()
SET #password = HashBytes('MD5', #password + CONVERT(VARCHAR(50),#Id))
SELECT #Password
INSERT INTO Employees.Registry (Id,[Name],[Password]) VALUES (#Id, #UserName,#Password)
END
Collation: SQL_Latin1_General_CP1_CI_AS
ProductVersion: 10.50.1600.1
Thanks
You are mixing 2 datatypes:
password need to be nvarchar to support non-Western European characters
literals need N prefix
Demo:
DECLARE #pwdgood nvarchar(150), #pwdbad varchar(150)
SET #pwdgood = N'ù9qÆæ\2 Ý-³Å¼]ó'
SET #pwdbad = N'?9q??\2 ?-³?¼]?'
SELECT #pwdgood, #pwdbad
HashBytes gives varbinary(8000) so you need this in the table
Note: I'd also consider salting the stored password with something other than ID column for that row
If you want to store such characters, you need to:
use NVARCHAR as the datatype for your columns and parameters (#Password isn't NVARCHAR and the CAST you're using to assign the password in the database table isn't using NVARCHAR either, in your sample ...)
use the N'....' syntax for indicating Unicode string literals
With those two in place, you should absolutely be able to store and retrieve any valid Unicode character
Due to a rather brilliant oversight in my current project, we have some guids getting stored in a varchar column in one table, which need to be compared to a uniqueidentifier column in another.
How can I do this? SQL server simply says it cannot convert from a character string to a uniqueidentifier.
If SQL complains it cannot cast it means not only you stored the uniqueidentifier as varchar, you used a different format than SQL Server (eg. you added the '{' and '}'). SQL is perfectly capable of casting string to uniqueidentifier when properly formatted:
declare #u uniqueidentifier;
declare #s varchar(64);
select #u = NEWID();
select #s = CAST(#u as varchar(64));
select CAST(#s as uniqueidentifier), #u, #s;
Depending on how you actualy stored the uniqueidentifier, you will most likely have tomodify the data and your code to match the SQL format (no {}).
Convert the uniqueidentifier to varchar:
CAST( uniqueidentifier_col_name as varchar)
I just worked up the following test script:
DECLARE
#Foo Uniqueidentifier
,#Foo2 varchar(50)
SET #Foo = newid()
SET #Foo2 = newId()
print #Foo
print #Foo2
if #Foo = #Foo2
print 'Yes'
else
print 'No'
set #Foo = #Foo2
if #Foo = #Foo2
print 'Yes'
else
print 'No'
Run in an SSMS window or via slqcmd -i file, the results are the same -- SQL (2005) does implicit conversion. This supports what I recall from SQL 2000 when I had a similar problem years ago.
The key thing is that the varchar string has to match the guid pattern:
8 hex digits
dash
4 hex digits
dash
4 hex digits
dash
4 hex digits
dash
12 hex digits
You'll have to cast the other uniqueidentifier to varchar.
SQL Server is probably tryng to cast things like "bob" to uniqueidentifier and it fails.
According to CAST/CONVERT it's allowed, so it must be the values in the varchar column.
I've got a stored procedure in my database, that looks like this
ALTER PROCEDURE [dbo].[GetCountingAnalysisResults]
#RespondentFilters varchar
AS
BEGIN
#RespondentFilters = '''8ec94bed-fed6-4627-8d45-21619331d82a, 114c61f2-8935-4755-b4e9-4a598a51cc7f'''
DECLARE #SQL nvarchar(600)
SET #SQL =
'SELECT *
FROM Answer
WHERE Answer.RespondentId IN ('+#RespondentFilters+'''))
GROUP BY ChosenOptionId'
exec sp_executesql #SQL
END
It compiles and executes, but somehow it doesn't give me good results, just like the IN statement wasn't working. Please, if anybody know the solution to this problem, help me.
You should definitely look at splitting the list of GUIDs into a table and joining against that table. You should be able to find plenty of examples online for a table-valued function that splits an input string into a table.
Otherwise, your stored procedure is vulnerable to SQL injection. Consider the following value for #RespondentFilters:
#RespondentFilters = '''''); SELECT * FROM User; /*'
Your query would be more secure parsing (i.e. validating) the parameter values and joining:
SELECT *
FROM Answer
WHERE Answer.RespondentId IN (SELECT [Item] FROM dbo.ParseList(#RespondentFilters))
GROUP BY ChosenOptionId
or
SELECT *
FROM Answer
INNER JOIN dbo.ParseList(#RespondentFilters) Filter ON Filter.Item = Answer.RespondentId
GROUP BY ChosenOptionId
It's slightly more efficient as well, since you aren't dealing with dynamic SQL (sp_executesql will cache query plans, but I'm not sure if it will accurately identify your query as a parameterized query since it has a variable list of items in the IN clause).
You need single quotes around each GUID in the list
#RespondentFilters = '''8ec94bed-fed6-4627-8d45-21619331d82a'', ''114c61f2-8935-4755-b4e9-4a598a51cc7f'''
It looks like you don't have closing quotes around your #RespondentFilters '8ec94bed-fed6-4627-8d45-21619331d82a, 114c61f2-8935-4755-b4e9-4a598a51cc7f'
Since GUIDs do a string compare, that's not going to work.
Your best bet is to use some code to split the list out into multiple values.
Something like this:
-- This would be the input parameter of the stored procedure, if you want to do it that way, or a UDF
declare #string varchar(500)
set #string = 'ABC,DEF,GHIJK,LMNOPQRS,T,UV,WXY,Z'
declare #pos int
declare #piece varchar(500)
-- Need to tack a delimiter onto the end of the input string if one doesn't exist
if right(rtrim(#string),1) ','
set #string = #string + ','
set #pos = patindex('%,%' , #string)
while #pos 0
begin
set #piece = left(#string, #pos - 1)
-- You have a piece of data, so insert it, print it, do whatever you want to with it.
print cast(#piece as varchar(500))
set #string = stuff(#string, 1, #pos, '')
set #pos = patindex('%,%' , #string)
end
Code stolen from Raymond Lewallen
I think you need quotes inside the string too. Try:
#RespondentFilters = '''8ec94bed-fed6-4627-8d45-21619331d82a'',''114c61f2-8935-4755-b4e9-4a598a51cc7f'''
You could also consider parsing the #RespondentFilters into a temporary table.
Tank you all for your ansewers. They all helped a lot. I've dealt with the problem by writing a split function, and it works fine. It's a litte bit overhead from what I could have done, but you know, the deadline is hiding around the corner :)