I have a Visual Studio Database project (SQL Server) with tables, stored procedures etc. The tables have collation defined ex:
CREATE TABLE [dbo].[TestTable]
(
[TestColumn] [varchar] (3) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL
);
The database default collation is also SQL_Latin1_General_CP1_CI_AS.
I use sqlpackage to publish and the ScriptDatabaseCollation set to True.
When I modify the table from any direction (like adding a new column), and use the SSDT compare tool, it shows the collation as a different, even though the "Ignore collation" is set to True:
Also, another interesting is that, when I click on the generate script, it won't contain any collation modifications, just the new column.
It's even worst when I try to compare from the other direction (update the DB directly and use compare from DB to local project), because it updates my file and removes the collation.
Sytem information:
SSDT Version 17.0.62204.01010
MSSQL Server Express 15.0.4153.1
Visual Studio Professional 2022 17.2.2
Does anybody know how can I solve this problem?
I can only presume that SSDT is thus trying to remove "excessive" DML which it thinks is unnecessary. Since your column's collation is the same as that of the project, repeating it again doesn't really make much sense (at least from SSDT's point of view).
You probably will appreciate this behaviour if / when you will have multiple instances of your database with different default collations. Speaking of which, I hope you know what you are doing, choosing a very old, problematic SQL collation as a default for your system.
Having said that, SSDT doesn't always remove collate clauses from your DML. If you specify a column collation which is different from the project's default one, it won't disappear after schema comparison (assuming both source and target have the same one). In one of my recent projects, for example, I needed a couple of columns to be case-sensitive, so I set them to Latin1_General_100_CS_AS in SSDT. These clauses didn't go anywhere after several months of development work.
If, for some unknown reason, it is absolutely paramount for you to keep these collate clauses in your code, you may set the project's default collation to something else. This will prevent SSDT from cleaning up the noise. However, you need to be careful with schema comparison and DACPAC deployment. In the former, you have the following options:
"Compare using target collation" (cleared by default),
"Ignore column collation" (cleared by default),
"Verify collation compatibility" (set by default).
(Not sure about the latter, as I never used it.)
However, going to the Schema Compare settings dialog every time you need to compare schemas will soon become too tedious to bear. I would recommend to just agree with SSDT and remove the stuff you don't really need.
Related
(Note that this was on SQL Server 2008, but I have a colleague who reports the same issue on SQL Server 2014.)
I'm using a framework that supports multiple database back-ends, and our application has a table called sysUsers, which works fine in MySQL.
We now need to install it on SQLServer and it appears that this name conflicts with a built-in system view. The system view is sys.sysusers and the application table is dbo.sysUsers.
I am aware that the case difference is irrelevant to SQL Server, however the schema seems to be being ignored for some reason.
SELECT * FROM sys.sysusers; returns records from sys.sysusers. This is wholly as expected.
SELECT * FROM sysUsers; returns records from sys.sysusers. This is surprising (I would have thought the local schema would take precedence) but perhaps explicable.
However, SELECT * FROM dbo.sysUsers; still returns records from sys.sysusers. This seems just plain wrong as I am explicitly selecting the dbo schema.
I haven't found anything in the MS documentation that says these names are reserved.
I have tried renaming the table and hacking the code to use a different name, and everything works (i.e. this is nothing to do with the SQLServer integration within the application) and the same results are seen when running the queries from the management console directly. Therefore this appears to definitely be an issue with the conflicting table name and not a middle-ware error or syntax difference.
If this table name is reserved, why does MSSMS allow me to create it? If it is not reserved, why does it not let me query it?
And how can I work round the problem without requiring application updates (as these would be a migration headache for other deployments).
There are at least three workarounds, but none guarantee that no code has to be rewritten (except the one that's horribly unsafe):
Use a case-sensitive collation when creating your database (CREATE DATABASE Foo COLLATE Latin1_General_CS_AS). In this case, sysUsers will be a different object from sysusers, in all circumstances. You can set a case-insensitive collation immediately after creating the database so your data doesn't end up case-sensitive, as this is probably not what the end users want. Obviously this won't work if your application is actually relying on case-insensitive object names, unless you rewrite your queries carefully. Note that this means that all database objects, even those created afterwards, will have case-sensitive names, as this is embedded in the system tables on creation and can't be changed afterwards.
Use a schema other than dbo. The system table mapping occurs only for that scheme, not any others. If your application uses its own schema exclusively, any sysusers you create in that will not be aliased to sys.sysusers. (This isn't documented anywhere, but it is how it works.) Note that in order for this to work, you must always specify the schema explicitly even when it is the default schema for your user, otherwise you will again get the system table (I'd consider this a bug, but it's probably a necessity because of the way old scripts will assume sysusers resolves anywhere).
Enable the Dedicated Administrator Connection, restart SQL Server in single user mode, switch the mssqlsystemresource database to READ_WRITE and DROP VIEW sysusers. This will remove sys.sysusers from all databases. Doing this will void your warranty, it will cause Microsoft Support to laugh at you if you come crying to them, it may make installing future Service Packs and updates impossible and is emphatically not recommended, but I'm mentioning it anyway, for science. No code anywhere should be using this view, but, you know, I'm not an engineer working on SQL Server itself.
Note that lowering the compatibility level is not a workaround, which I mention for completeness. This has no effect on how these table names are resolved, even if it was a desirable approach (which it's not).
I consider the change made in SQL Server 2012 to ignore the dbo qualifier and resolve to these old, deprecated names anyway a mistake and if it were up to me I'd at least make it possible to opt out of this behavior with a trace flag, but it's not up to me. You could consider opening up an issue on Microsoft Connect for it, because the current behavior makes it needlessly complicated for RDBMS-agnostic code to run.
According to the SonarQube documentation "Installing the Server" (https://docs.sonarqube.org/display/sonar/installing+the+server), for a Microsoft SQL Server host, "collation MUST be case-sensitive (CS) and accent-sensitive (AS)."
The documentation is not clear if the collation must be set:
for the SQL Server instance, or
the database
If the collation for the SQL Server (and specifically for tempdb) is "accent insensitive" and the database collation is "accent sensitive", does SonarQube accommodate this configuration?
If the collation for the SQL Server (and specifically for tempdb) is "accent insensitive" and the database collation is "accent sensitive", does SonarQube accommodate this configuration?
Since the documentation is ambiguous (they might not use SQL Server enough to know the different levels where Collation can be set), the only two ways to get the answer here are:
Contact their community: https://www.sonarqube.org/community/feedback/. This is the best choice.
Install it on an Instance that has an accent insensitive default Collation and test it out. No reason not to try this.
Whether or not SonarQube handles this properly depends on how it was coded. They could be JOINing on string columns in temporary tables and any difference in Collation between the Database and Instance could potentially cause an error, but only if they are not specifically declaring the Collation when creating the temp tables.
Also, it is possible that their app needs the accent sensitivity because they have some variables names and/or cursor names and/or (less likely) GOTO label names that might equate under accent insensitivity that should otherwise be seen as different. Instance-level Collation controls these areas and would hence affect the name resolution of those items. Of course, this would be easy to test for since declaring two variables that are considered different names under accent sensitivity will cause a parse error if close enough to be considered the same under accent insensitivity. Still, contact their community.
Dear all, Currently I am just researching how I could handle the change of the collation on the database.
Somebody made an unusual decision to create accent sensitive database for global use... but I am on the way to handle this!
REASON: of changing the collation is that database contains data collected from different countries and as we all know some of cultures have their own letters.
With the respect for the customers, our organization would like to have Accent Insensitive database. That will allow users to request data from the server without any limitations using local characters.
As far as I have find out, there may be an option to drop constraints and etc. change collation and then just to bring everything back. In this case I am afraid if this would be enough to affect already existing data (columns).
Another way, I have found an article in Collation change on 2005 and 2008 server. However, this does not include the 2012 server.
Also I am taking the complexity of this example into consideration as well.
I believe that I am not in an easy phase. But I am hoping to get few advises what would be the best and safest way to handle this.
Thank you for your concerns and assistance.
UPDATE let me add what architecture do we have: The complete system contains 4 databases and more than 1.000 tables in total. So my expectations is that not all of the possible ways may work in an optimal way.
me too i had to deal with a similar issue because of a different reason: ancient databases with an old SQL collation installed ages ago on a SQL6.5 server that has been inplace upgraded for each version from sql 7 to sql 2005 and now should be updated to sql 2012.
why all these inplace upgrades? because the actual collation was the server collation and was so old that is not available during then install process of a recent version (2000+) of sql server...
i decided to drop all that old rubbish so i had to find a way that allowed me to move to a new installation with a windows collation.
i had to exclude the data migration (create a new database and import data) because of the lack of documentation and the huge number of customizations, triggers, hidden rules and so on.
the solution i used (the order matters):
disable automatic statistics generation
script the creation of all foreign keys and then drop them
script unique and primary indexes and then drop them
script all remaining indexes and then drop them
script custom statistics and then drop them
script CHECK and DEFAULT constraints and then drop them
now you can run the ALTER commands needed to change the collation of the columns and change the collation of the database itself.
when done repeat the above in reverse order to rebuild all the needed objects.
it happens that if the database is so old as is mine you may incur in something funny like existing foreign key that references fields with different datatypes.
Changing collation of all existing columns is a real pain. I suggest a side-by-side migration rather than alter each column individually. Create a new database with the desired collation containing only empty tables. Copy data from the old db to the new one using INSERT...SELECT (or the ETL tool of your choice), and then create constraints, indexes, and other database objects.
Consider upvoting the Make it easy to change collation on a database SQL Server feature request.
There are a number of complicated solutions on the internet for inplace collation changes but the simplest (and safest) way we have found is to script out the database, alter the script to create a new db with the collation set at the start and then import the data to the new database.
We achieve this using MS SQL Server 2012 Management Studio in the following way:
Script out all database objects with Tasks -> Generate Scripts -> Script entire Database and all Database objects
Alter the script with the following 2 changes and then run it to create a new database:
a) Change DB name to MY-NEW-DB
b) Under the CREATE DATABASE statement add: ALTER DATABASE [MY-NEW-DB] collate Latin1_General_CS_AS
If desired, use a tool like RG SQL Compare to compare the old and new database to verify all indexes, constraints, types etc were the same and collation on relevant columns only was changed.
Run Tasks->Import Data ensuring 'Enable Identity Insert' checked. All data transferred to the new case sensitive database correctly.
Run DBCC CHECKDB if you wish to check consistency
In SQL Server (2000 or 2005) is it possible to set the database or server collation so that identifier names (tables, columns, etc) need to be in the correct case? If so, is that true of all case-sensitive collations or is it a separate setting? (I've always thought of case-sensitivity applying to data, not to names of objects).
Presumably this would break an application if its stored procs and queries weren't written with consistent case? Is there a way to deal with this without having to ensure all queries use the correct case, such as setting the collation of a database connection?
I'm looking at this from the point of view of having an existing application which probably has inconsistently cased sql code in it, and I'm wanting to be able to run it against databases with different collations. What settings would I need or what set of database/server collations could I not use the application with?
The collation is what determines if your queries will be case insensitive. So the only way to ensure that your schema will work against multiple environments is to have your queries be case sensitive. If your queries are not consistent, then your collation MUST be case insensitive otherwise it will not work.
http://msdn.microsoft.com/en-us/library/aa174903(SQL.80).aspx
One thing to note is that once you've set up your SQL Server environment with a certain collation, you CANNOT change it without creating a NEW SQL Server instance. So Case-Insensitive is usually the way to go. And then strive to have consistency in your queries.
Once a collation is set it applies to both data and metadata, I believe.
Collation is set in earlier versions of SQL Server, but in 2005 and beyond, you can change it by object, as they are created.
The database default collation determines whether objects within the database are treated in a case-sensitive way in queries - this applies to all object name: tables, columns, etc.
If your application code comes from a case-insensitive collation database, it may not run on a case-sensitive collation database if a object is misreferenced (you would get a message when you attempted to run the statement or create the stored procedure, or in a stored-proc architecture, you would catch all these pretty quickly unless you had a significant amount of dynamic SQL).
Remember, that even if your code runs, individual columns can be set with collations which differ from the database, so it's always possible that with a differing collation, your code will behave unexpectedly (for instance, GROUP BY behaves differently).
You can set collation for each object, and set a default for the database and server as well.
How to deal with it? You need to enforce standards here. You can easily get yourself tangled up with different people write with different case.
The collation also applies to data so "bob" != "Bob"
Transferring data from one SQL server to another but when Schema is compared and syncronised the following error is received. We are using redgate SQL compare to complete.
Cannot resolve collation conflict for equal to operation
Base SQL server is SQL_Latin1_General_CP1_CI_AS and the destination server is Latin1_General_CI_AS
SQL Compare has an option to ignore collations. Look under the tab "options" in your compare project configuration.
is you problem with the SQL Compare utlity, or a worry that different server collations will lead to problems?
You could change the collation of the destination server to match the Base server
If that is not possible, then make the Collation of the databases on each server match, and then your only real problem is likely to be any temporary tables which you create (they will have a default collation matching the server / TEMPDB), and so long as you explicitly create the temporary table (i.e. don't create it using SELECT * INTO #TEMP FROM MyTable) and explicitly assign a collation to any varchar/text columns you should be OK
The way I overcome this is to generate the scripts via SQL Compare and then strip out (or replace) the Collation specific code. This is relatively fast and easy to do, and finally I manually apply the scripts to the destination server/ database.
Sounds like the collation settings for the server are different.
How are you transferring the data, do you perform a database restore on your new platform?
Either way, you need to ensure that the same collation is used on your new environment as is currently in place in your source environment.
Hope this makes sense, let me know if you need further assistance.
"Ignore collations" is definitely not going to work, for the reason stated above. The problem happens when migrating objects like views and stored procedures that use JOIN clauses on text fields that have differing collations.
If someone changes the default collation on the server and the column on the other side of the JOIN uses a specific collation, you've caused this issue. And it would happen in SQL Compare as well as if you just manually scripted the object in SSMS and moved it yourself.
There are two roads to fixing it - you could specify a COLLATE clause on the join and explicitly state the collation you want to use, or you could change the destination database default collation to match the source.
I'm afraid there is no SQL Compare "magic bullet" to solve this.