I have a database that is supposed to store data in any language, there is going to be a column that tells me which locale it is, so i can't rely on database collation and will have to specify collation at runtime in queries.
I also have the problem that i want to use EF for dataaccess, as we know using EF one cannot specify collation at runtime. I am thinking about creating a sql function that takes collation as argument and apply that function in all of the Linq Queries.
but this fails
CREATE FUNCTION fn_Compare
(
#TextValue nvarchar(max),
#Culture varchar(10)
)
RETURNS nvarchar(max)
AS
BEGIN
RETURN #TextValue COLLATE #Culture
END
GO
does anyone know if this can be done ?
You cannot do this. The collation returned by the function needs to be consistent across all the return values. For instance, the following generates an error:
create function testfn (#test varchar(100), #i int)
returns varchar(100)
as
begin
return(case when #i = 0 then #test collate SQL_Latin1_General_CP1_CS_AS
else #test collate SQL_Latin1_General_CP1_CI_AS
end)
end;
The error is due to a collation conflict.
What you can do is use:
alter database collate <whatever>
Or, alternatively, create a new working database with the collation you want.
Related
I've searched high and low but can't find an answer, can you set the collation of a variable? According to the MS documentation, it seems that it's only possible on SQL Azure:
-- Syntax for Azure SQL Data Warehouse and Parallel Data Warehouse
DECLARE
{{ #local_variable [AS] data_type } [ =value [ COLLATE ] ] } [,...n]
Currently I have to do this:
DECLARE #Test nvarchar(10) = N'Crud';
IF ( #Test = N'Crud' COLLATE Latin1_General_CS_AI )
Print N'Crud';
IF ( #Test = N'cRud' COLLATE Latin1_General_CS_AI )
Print N'cRud';
IF ( #Test = N'crUd' COLLATE Latin1_General_CS_AI )
Print N'crUd';
IF ( #Test = N'cruD' COLLATE Latin1_General_CS_AI )
Print N'cruD';
When what I'd like to do is this:
DECLARE #Test nvarchar(10) = N'Crud' COLLATE Latin1_General_CS_AI;
IF ( #Test = N'Crud' )
Print N'Crud';
IF ( #Test = N'cRud' )
Print N'cRud';
IF ( #Test = N'crUd' )
Print N'crUd';
IF ( #Test = N'cruD' )
Print N'cruD';
I'm guessing the answer is no but I wanted to confirm and at the very least, someone else ever needing this info will get a definitive answer.
Much appreciated.
Well, you're guessing correctly.
In most SQL Server systems, (meaning, not including Azure SQL Data Warehouse and Parallel Data Warehouse) A collation can be set on four levels:
The default collation of the SQL Server instance:
The server collation acts as the default collation for all system databases that are installed with the instance of SQL Server, and also any newly created user databases.
The default collation of a specific database:
You can use the COLLATE clause of the CREATE DATABASE or ALTER DATABASE statement to specify the default collation of the database. You can also specify a collation when you create a database using SQL Server Management Studio. If you do not specify a collation, the database is assigned the default collation of the instance of SQL Server.
You can set a collation for a table's column:
You can specify collations for each character string column using the COLLATE clause of the CREATE TABLE or ALTER TABLE statement. You can also specify a collation when you create a table using SQL Server Management Studio. If you do not specify a collation, the column is assigned the default collation of the database.
You can set a collation for a specific expression using the Collate clause:
You can use the COLLATE clause to apply a character expression to a certain collation. Character literals and variables are assigned the default collation of the current database. Column references are assigned the definition collation of the column.
So yes, with the exception of Azure SQL Data Warehouse and Parallel Data Warehouse, you can't set a collation on a local scalar variable.
When I SELECT from the table, the data that I stored is stored as question marks.
#word is a parameter in my stored procedure, and the value comes from the C# code:
string word = this.Request.Form["word"].ToString();
cmd.Parameters.Add("#word", System.Data.SqlDbType.NVarChar).Value = word;
My stored procedure is like this:
CREATE PROCEDURE ....
(
#word nvarchar(500)
...
)
Insert into rub_translate (language_id,name)
values (8 ,#word COLLATE HEBREW_CI_AS )
My database, and the column, is using the SQL_Latin1_General_CP1_CI_AS collation and I cannot change them.
Can anybody give me a solution how can I solve this problem just by modifying the column or the table?
In order for this to work you need to do the following:
Declare the input parameter in the app code as NVARCHAR (you have done this)
Declare the input parameter in the stored procedure as NVARCHAR (no code is shown for this)
Insert or Update a column in a table that is defined as NVARCHAR (you claim that this is the case)
When using NVARCHAR it does not matter what the default Collation of the Database is. And actually, when using NVARCHAR, it won't matter what the Collation of the column in the table is, at least not for properly inserting the characters.
Also, specifying the COLLATE keyword in the INSERT statement is unnecessary and wouldn't help anyway. If you have the stored procedure input parameter defined as VARCHAR, then the characters are already converted to ? upon coming into the stored procedure. And if the column is actually defined as VARCHAR (you haven't provided the table's DDL) then if the Collation isn't Hebrew_* then there is nothing you can do (besides change either the datatype to NVARCHAR or the Collation to a Hebrew_ one).
If those three items listed at the top are definitely in place, then the last thing to check is the input value itself. Since this is a web app, it is possible that the encoding of the page itself is not set correctly. You need to set a break point just at the cmd.Parameters.Add("#word", System.Data.SqlDbType.NVarChar).Value = word; line and confirm that the value held in the word variable contains Hebrew characters instead of ?s.
ALSO: you should never create a string parameter without specifying the max length/size. The default is 30 (in this case, sometimes it's 1), yet the parameter in the stored procedure is defined as NVARCHAR(500). This could result in silent truncation ("silent" meaning that it will not cause an error, it will just truncate the value). Instead, you should always specify the size. For example:
cmd.Parameters.Add("#word", System.Data.SqlDbType.NVarChar, 500).Value = word;
You could just insert it as-is, since it's unicode and then select it with a proper collation:
declare #test table([name] nvarchar(500) collate Latin1_General_CI_AS);
declare #word nvarchar(500) = N'זה טקסט.';
insert into #test ( [name] ) values ( #word );
select [t].[name] collate Hebrew_CI_AS from #test as [t]
Or you can change the collation of that one column in the table all together. But remember that there is a drawback of having a different collation from your database in one or more columns: you will need to add the collate statement to queries when you need to compare data between different collations.
I'm storing encrypted passwords in the database, It worked perfect so far on MachineA. Now that I moved to MachineB it seems like the results gets corrupted in the table.
For example: ù9qÆæ\2 Ý-³Å¼]ó will change to ?9q??\2 ?-³?¼]? in the table.
That's the query I use:
ALTER PROC [Employees].[pRegister](#UserName NVARCHAR(50),#Password VARCHAR(150))
AS
BEGIN
DECLARE #Id UNIQUEIDENTIFIER
SET #Id = NEWID()
SET #password = HashBytes('MD5', #password + CONVERT(VARCHAR(50),#Id))
SELECT #Password
INSERT INTO Employees.Registry (Id,[Name],[Password]) VALUES (#Id, #UserName,#Password)
END
Collation: SQL_Latin1_General_CP1_CI_AS
ProductVersion: 10.50.1600.1
Thanks
You are mixing 2 datatypes:
password need to be nvarchar to support non-Western European characters
literals need N prefix
Demo:
DECLARE #pwdgood nvarchar(150), #pwdbad varchar(150)
SET #pwdgood = N'ù9qÆæ\2 Ý-³Å¼]ó'
SET #pwdbad = N'?9q??\2 ?-³?¼]?'
SELECT #pwdgood, #pwdbad
HashBytes gives varbinary(8000) so you need this in the table
Note: I'd also consider salting the stored password with something other than ID column for that row
If you want to store such characters, you need to:
use NVARCHAR as the datatype for your columns and parameters (#Password isn't NVARCHAR and the CAST you're using to assign the password in the database table isn't using NVARCHAR either, in your sample ...)
use the N'....' syntax for indicating Unicode string literals
With those two in place, you should absolutely be able to store and retrieve any valid Unicode character
I have problem compilin this code..can anyone tell whats wrong with the syntax
CREATE PROCEDURE spGenericInsert
(
#insValueStr nvarchar(200)
#tblName nvarchar(10)
)
AS
BEGIN
DECLARE #insQueryStr nvarchar(400)
DECLARE #insPrimaryKey nvarchar(10)
DECLARE #rowCountVal integer
DECLARE #prefix nvarchar(5)
IF #tblName='HW_Master_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM HW_Master_DB)
ELSE IF #TableName='SW_Master_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM SW_Master_DB)
ELSE IF #TableName='INV_Allocation_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM INV_Allocation_DB)
ELSE IF #TableName='REQ_Master_DB'
SET #rowCountVal=(SELECT COUNT(*) FROM REQ_Master_DB)
IF #tblName = 'DEFECT_LOG'
SET #prefix='DEF_'
ELSE IF #tblName='INV_Allocation_DB'
SET #prefix='INV_'
ELSE IF #tblName='REQ_Master_DB'
SET #prefix='REQ_'
ELSE IF #tblName='SW_Master_DB'
SET #prefix='SWI_'
ELSE IF #tblName='HW_Master_DB'
SET #prefix='HWI_'
SET #insPrimaryKey= #prefix + RIGHT(replicate('0',5)+ convert(varchar(5),#rowCountVal),5) -- returns somethin like 'DEF_00005'
SET #insQueryStr= 'INSERT INTO ' + #tblName + ' VALUES (' + #insPrimaryKey + ',' + #insValueStr + ')'
EXEC(#insQueryStr)
END
I know about Integer Identity columns.. but i have to use a AlphaNumeric ID in the tables in inserting new values in a highly multi-user intranet system.
The records will not be deleted from the table. So problem is that of maintain synchronous insertion of records with ID field automatically generated.
Any suggestions how that can be done.
Take your pick:
#TableName isn't defined
#tblName vs. #TableName
I cannot immediately see what's wrong with the syntax (the sharp eye of Jonathan Lonowski has solved that already), but there are some things wrong with the code:
You create dynamic SQL, so your code is vunerable to SQL-injection attacks. Both the input parameters are used in a dangerous way. Solve this by creating a stored procedure for every table. So you don't have to generate SQL anymore.
There is no check if the table is not in the list used.
Your primary key generation algorithm can/will create duplicate keys in a multi-user scenario, or if rows are deleted from the table. Solve by using an identity column or some other feature from the database you are using.
Honestly, you seem to be making a headache for yourself. Check out integer identities and IDENTITY syntax.
Unless you are truly required to use keys in the "DEF_00005" format, they will make your life a lot easier.
CREATE TABLE DemoTable (
Key INT IDENTITY(1,1) NOT NULL PRIMARY KEY,
Value VARCHAR(200)
);
INSERT INTO DemoTable (Value) VALUES ('Something');
SELECT * FROM DemoTable;
| Key | Value |
|-----|-----------|
| 1 | Something |
Aside from missing lots of semicolons, you're going to have to give us more to go on.
Actually, SQL Server might not need semicolons, so ignore that...
But here is a good place to start learning about stored prcedures in SQL server. You can search Google for some more as well.
How can I ignore accents (like ´, `, ~) in queries made to a SQL Server database using LINQ to SQL?
UPDATE:
Still haven't figured out how to do it in LINQ (or even if it's possible) but I managed to change the database to solve this issue.
Just had to change the collation on the fields I wanted to search on. The collation I had was:
SQL_Latin1_General_CP1_CI_AS
The CI stans for "Case Insensitive" and AS for "Accent Sensitive". Just had to change the AS to AI to make it "Accent Insensitive".
The SQL statement is this:
ALTER TABLE table_name ALTER COLUMN column_name column_type COLLATE collation_type
In SQL queries (Sql Server 2000+, as I recall), you do this by doing something like select MyString, MyId from MyTable where MyString collate Latin1_General_CI_AI ='aaaa'.
I'm not sure if this is possible in Linq, but someone more cozy with Linq can probably translate.
If you are ok with sorting and select/where queries ALWAYS ignoring accents, you can alter the table to specify the same collation on the field(s) with which you are concerned.
See the following answer:
LINQ Where Ignore Accentuation and Case
Basically you need to alter the field type in SQL Server, e.g.
ALTER TABLE People ALTER COLUMN Name [varchar](100) COLLATE SQL_Latin1_General_CP1_CI_AI
There does not seem to be a way to do this using LINQ, apart from calling a custom method to remove diacritics (which would not be performant).
LINQ to SQL doesn't have any specific functionality for setting the collation used for a query and so it will always use the database default.
It seems that there is a way to ignore the collation differences in Linq to SQL by using t-sql functions:
CREATE FUNCTION [dbo].[func_ConcatWithoutCollation]
(
#param1 varchar(2000),
#param2 varchar(2000)
)
RETURNS varchar(4000)
AS
BEGIN
IF (#param1 IS NULL) SET #param1 = ''
IF (#param2 IS NULL) SET #param2 = ''
RETURN #param1 COLLATE Latin1_General_CS_AS + #param2 COLLATE Latin1_General_CS_AS
END
to get this function in linq to sql, there is a switch for SqlMetal: /functions.
Example:
"%ProgramFiles%\Microsoft SDKs\Windows\v7.0A\Bin\SqlMetal.exe" /server:. /database:NameOfDatabase /pluralize /code:ContextGenerated.cs /sprocs /views /functions
Use this function in Linq to sql like this:
from s in context.Services
where context.Func_ConcatWithoutCollation(s.Description, s.Email) == "whatever"
select s
It helped me, maybe somebody finds this useful too.
A solution could be create an SQL Function to remove the diacritics, by applying to the input string the collation SQL_Latin1_General_CP1253_CI_AI, like so:
CREATE FUNCTION [dbo].[RemoveDiacritics] (
#input varchar(max)
) RETURNS varchar(max)
AS BEGIN
DECLARE #result VARCHAR(max);
select #result = #input collate SQL_Latin1_General_CP1253_CI_AI
return #result
END
Then you could add it in the DB context (in this case ApplicationDbContext) by mapping it with the attribute DbFunction:
public class ApplicationDbContext : IdentityDbContext<CustomIdentityUser>
{
[DbFunction("RemoveDiacritics", "dbo")]
public static string RemoveDiacritics(string input)
{
throw new NotImplementedException("This method can only be used with LINQ.");
}
public ApplicationDbContext(DbContextOptions<ApplicationDbContext> options)
: base(options)
{
}
}
And finally use it in LINQ query, for example (linq-to-entities):
var query = await db.Users.Where(a => ApplicationDbContext.RemoveDiacritics(a.Name).Contains(ApplicationDbContext.RemoveDiacritics(filter))).tolListAsync();