Task:
Automate database deployment (SSDT/dacpac deployment with CI/CD)
The database is a 3rd party database
It also includes our own customized tables/SP/Fn/Views in separate schemas
Should exclude 3rd party objects while deploying the database project(dacpac) to Production
Thanks to Ed Elliott for the AgileSqlClub.DeploymentFilterContributor. Used the dll to filter out the schema successfully.
Problem:
The 3rd party schema objects(Tables) are defined with unnamed constraints(default / primary key) when creating the tables. Example:
CREATE TABLE [3rdParty].[MainTable]
(ID INT IDENTITY(1,1) NOT NULL,
CreateDate DATETIME DEFAULT(GETDATE())) --There is no name given to default constraint
When I generate the script for deployment using sqlpackage.exe, I see following statements in the generated script.
Generated the script using:
"C:\Program Files\Microsoft SQL Server\150\DAC\bin\sqlpackage.exe" /action:script /sourcefile:C:\Users\User123\source\repos\DBProject\DBProject\bin\Debug\DBProject.dacpac /TargetConnectionString:"Data Source=MyServer; Initial Catalog=MSSQLDatabase; Trusted_Connection=True" /p:AdditionalDeploymentContributorPaths="C:\Program Files\Microsoft SQL Server\150\DAC\bin\AgileSqlClub.SqlPackageFilter.dll" /p:AdditionalDeploymentContributors=AgileSqlClub.DeploymentFilterContributor /p:AdditionalDeploymentContributorArguments="SqlPackageFilter=IgnoreSchema(3rdParty)" /outputpath:"c:\temp\script_AfterDLL.sql"
Script Output:
/*
Deployment script for MyDatabase
This code was generated by a tool.
Changes to this file may cause incorrect behavior and will be lost if
the code is regenerated.
*/
...
...
GO
PRINT N'Dropping unnamed constraint on [3rdParty].[MainTable]...';
GO
ALTER TABLE [3rdParty].[MainTable] DROP CONSTRAINT [DF__MainTabl__Crea__59463169];
...
...
...(towards the end of the script)
ALTER TABLE [3rdParty].[MainTable_2] WITH CHECK CHECK CONSTRAINT [fk_518_t_44_t_9];
I cannot alter 3rd party schema due to company restrictions
There are many lines of unnamed constraint and WITH CHECK CHECK constraints generated in the script.
Question:
How can I be able to remove the lines to DROP unnamed Constraint on 3rd party schemas? - Even though the dll excludes 3rd party schema, it still has these unnamed constraints scripted/deployed. Also, it is not Adding them back too !!
How can I be able to skip/remove generating WITH CHECK CHECK CONSTRAINT on 3rd party schemas
Any suggestions will be greatly helpful.
EDIT:
Also, I found another issue. The deployment will not succeed due to Rows were detected. The schema update is terminating because data loss might occur
Output:
/*
The column [3rdParty].[MainTable_1].[Col1] is being dropped, data loss could occur.
The column [3rdParty].[MainTable_1].[Col2] is being dropped, data loss could occur.
The column [3rdParty].[MainTable_1].[Col3] is being dropped, data loss could occur.
The column [3rdParty].[MainTable_1].[Col4] is being dropped, data loss could occur.
*/
IF EXISTS (select top 1 1 from [3rdParty].[MainTable_1])
RAISERROR (N'Rows were detected. The schema update is terminating because data loss might occur.', 16, 127) WITH NOWAIT
GO
Regarding the unnamed constraints, I couldn't find any solution using sqlpackage.exe.
But Redgate SQL Compare has an option to ignore them called IgnoreSystemNamedConstraintAndIndexNames that ignores system generated constraints and generates a much cleaner script.
For example when comparing 2 dacpacs:
SQLCompare /Scripts1:"\unpacked_dacpac_source_folder" /Scripts2:"\unpacked_dacpac_dest_folder" /options:IgnoreSystemNamedConstraintAndIndexNames /scriptFile:"script_result.sql"
You can find more info here:
Handling System-named Constraints in SQL Compare
Background
I'm looking into creating a simple web app, a part of which will display Images associated with Items. I've decided to look into using the FILETABLE feature of SQL Server which will allow binary image data to be uploaded into the exposed share directly. As such there is a use case to allow the deletion of files (rows in a FILETABLE) through Windows Explorer. This example replicates the issue, which stems from having a foreign key relationship to a FILETABLE.
Structure
Having already added an image using File Explorer to the FILETABLE with the path_locator of 0xFF5354649088A1EFEE8F747CD11030F80800170620:
CREATE TABLE [dbo].[Image] AS FILETABLE WITH (FileTable_Directory = 'Images');
GO
CREATE TABLE [dbo].[ImageLink] (
[id] INT NOT NULL IDENTITY(1, 1)
,[path_locator] HIERARCHYID NOT NULL
,FOREIGN KEY ([path_locator]) REFERENCES [dbo].[Image] ([path_locator])
);
GO
INSERT INTO [dbo].[ImageLink] ([path_locator]) VALUES (0xFF5354649088A1EFEE8F747CD11030F80800170620);
Issue
Upon deleting the file through File Explorer...
... the file disappears from the directory as Windows reports the deletion a success but the row is not removed from the FILETABLE.
However, when trying to delete through SQL Server, the familiar reference constraint conflict error is thrown:
DELETE FROM [dbo].[Image] WHERE [path_locator] = 0xFF5354649088A1EFEE8F747CD11030F80800170620;
Msg 547, Level 16, State 0, Line 69
The DELETE statement conflicted with the REFERENCE constraint "FK__ImageLink__path___5070F446". The conflict occurred in database "FileTableTest", table "dbo.ImageLink", column 'path_locator'.
I added an AFTER DELETE trigger to the FILETABLE with the intention of removed the referencing row, but this also does not get executed.
Question
How might I go about propagating the delete through the link table upon deletion through Windows Explorer?
Is there some kind of SQL Server/Windows API hook I can detect and execute DML code that handles the delete?
Update #1
From BOL, the following section kind of confirms the behaviour, although doesn't offer any further information.
Transactional Semantics
When you access the files in a FileTable by using file I/O APIs, these operations are not associated with any user transactions, and have the following additional characteristics:
Since non-transacted access to FILESTREAM data in a FileTable is not associated with any transaction, it does not have any specific isolation semantics. However SQL Server may use internal transactions to enforce locking or concurrency semantics on the FileTable data. Any internal transactions of this type are done with read-committed isolation.
The problem is the foreign key.
Use 'ON CASCADE DELETE' in your foreign key, so when you delete through File Explorer the associated ImageLink is deleted too.
It looks like ,there is problem with the foreign key. As there is foreign key attached to that table so you cannot simply delete the row as foreign key contraint fails.
So first disable foreign key check in sql by :
SET FOREIGN_KEY_CHECKS = 1;
and then try deleting this & yes don't forget to set foreign key check to 0 by:
SET FOREIGN_KEY_CHECKS = 0;
after deleting the row.
I am using a SQL Server database with SQL Server Management Studio where I have existing tables. I add a few tables to it and it works just fine. However, for subsequent operations such as
Drop table XXX --OR
Create Trigger YYY on XXX
I run into a error statement that reads:
i) Cannot drop table XXX as it does not exist or you do not have permissions
ii) The object 'XXX' does not exist or is invalid for this operation
I tried to carry out an Insert operation but that showed me a similar error (The object 'XXX' does not exist). I can see this maybe a permissions issue since I am using an existing database. However, in that case, I should have been unable to create a table as well?
Can anyone pinpoint how I can work myself around this and what the problem is?
What is your default schema?
SELECT name, default_schema_name
FROM sys.database_principals
WHERE type = 'S';
Try qualifying your references to the table as SchemaName.XXX and see if that helps.
Most of times when I had similar situations tables were created in system databases (master, tempdb..). Of course it was my mistake.
So maybe try to search for a tables in other databases?
I have a database and have a Sql script to add some fields to a table called "Products" in the database.
But when i am executing this script, I am getting the following error:
Cannot find the object "Products" because it does not exist or you do not have permissions
Why is the error occurring and what should I do to resolve it?
I found a reason why this would happen. The user had the appropriate permissions, but the stored procedure included a TRUNCATE statement:
TRUNCATE TableName
Since TRUNCATE deletes items without logging, you (apparently) need elevated permissions to execute a stored procedure that contains it. We changed the statement to:
DELETE FROM TableName
...and the error went away!
Are you sure that you are executing the script against the correct database? In SQL Server Management studio you can change the database you are running the query against in a drop-down box on one of the toolbars, or you can start your query with this:
USE SomeDatabase
It can also happen due to a typo in referencing a table such as [dbo.Product] instead of [dbo].[Product].
Does the user you're executing this script under even see that table??
select top 1 * from products
Do you get any output for this??
If yes: does this user have the permission to modify the table, i.e. execute DDL scripts like ALTER TABLE etc.? Typically, regular users don't have this elevated permissions.
Look for any DDL operation in the script.
Maybe the user does not have access rights to run changes.
In my case it was SET IDENTITY_INSERT tblTableName ON
You can either add db_ddladmin for the whole database or for just the table to solve this issue (or change the script)
-- give the non-ddladmin user INSERT/SELECT as well as ALTER:
GRANT ALTER, INSERT, SELECT ON dbo.tblTableName TO user_name;
It could also be possible that you have created the "Products" in your login schema and you were trying to execute the same in a different schema (probably dbo)
Steps to resolve this issue
1)open the management studio
2) Locate the object in the explorer and identify the schema under which your object is? ( it is the text before your object name ). In the image below its the "dbo" and my object name is action status
if you see it like "yourcompanydoamin\yourloginid" then you should
you can modify the permission on that specific schema and not any other schema.
you may refer to "Ownership and User-Schema Separation in SQL Server"
I've been trying to copy a table from PROD to DEV but get an error:
"Cannot find the object X because it does not exist or you do not have permissions."
However, the table did exist, and I was running as sa so I did have permissions.
The problem was actually with CONTRAINTS. I'd renamed the table on DEV to be old_XXX months ago. But when I tried to copy the original one over from PROD, the Defaut Constraint names clashed.
The error message was misleading
You can right click the procedure, choose properties and see which permissions are granted to your login ID. You can then manually check off the "Execute" and alter permission for the proc.
Or to script this it would be:
GRANT EXECUTE ON OBJECT::dbo.[PROCNAME]
TO [ServerInstance\user];
GRANT ALTER ON OBJECT::dbo.[PROCNAME]
TO [ServerInstance\user];
This could be a permission issue. The user needs at least ALTER permission to truncate a table.
Another option is to call DELETE FROM instead of TRUNCATE TABLE, but this operation is slower because it writes to the Log file, whereas TRUNCATE does not write to the log file.
The minimum permission required is ALTER on table_name. TRUNCATE TABLE
permissions default to the table owner, members of the sysadmin fixed
server role, and the db_owner and db_ddladmin fixed database roles,
and are not transferable. However, you can incorporate the TRUNCATE
TABLE statement within a module, such as a stored procedure, and grant
appropriate permissions to the module using the EXECUTE AS clause.
Sharing my case, hope that will help.
In my situation inside MY_PROJ.Database->MY_PROJ.Database.sqlproj I had to put this:
<Build Include="dbo\Tables\MyTableGeneratingScript.sql" />
In my case I was running under a different user than the one I was expecting.
My code passed 'DRIVER={SQL Server};SERVER=...;DATABASE=...;Trusted_Connection=false;User Id=XXX;Password=YYY' as the connection string to pypyodbc.connect(), but it ended up connecting with the credentials of the Windows user that ran the script instead of the User Id= from the connection string.
(I verified this using the SQL Server Profiler and by putting an invalid uid/password combination in the connection string - which didn't result in an expected error).
I decided not to dig into this further, since switching to this better way of connecting fixed the issue:
conn = pypyodbc.connect(driver='{SQL Server}', server='servername',
database='dbname', uid='userName', pwd='Password')
In my case the sql server version on my localhost is higher than that on the production server and hence some new variables were added to the generated script from the localhost. This caused errors in creating the table in the first place.
Since the creation of the table failed, subsequent query on the "NON EXISITING" table also failed.
Luckily, in among the long list of the sql errors, I found this "OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF" to be the new varialbe in the script causing my issue. I did a search and replace and the error went away.
Hope it helps someone.
The TRUNCATE statement was my first problem, glad to find the solution here. But I was using SSIS and trying to load data from another database, and it failed with the same error on any table that used IDENTITY to create an auto-incrementing ID. If I was scripting it myself I'd first need to use the command SET IDENTITY_INSERT tablename ON, and then SET IDENTITY_INSERT tablename OFF when the table update was done. But this requires ALTER permissions on the table, which I do not have. Hence the error message in SSIS on the table load (even though the previous step had just deleted all the data out of the table.)
You receive this error, when you use an ORM like GORM (https://gorm.io/) in Go for example.
When you try to create a struct and accidentally pass the ID (primary key) although it's inserted automatically.
Rich features IDE like Visual Studio Code make this mistake happen easily:
if tx := db.Create(&myStruct{
Ts: Time.Now(),
ID: 42,
}); tx.Error != nil {
t.Fatal(tx.Error)
}
You can still use auto-filling by Visual Studio Code, but delete your entry for your model's primary keys:
if tx := db.Create(&myStruct{
Ts: Time.Now(),
}); tx.Error != nil {
t.Fatal(tx.Error)
}
when am going to insert data into a child table for which i have added constraint(foreign key)
the constraint is automatically disabling.
can u please help me..........
From the Utilties manual, relational integrity and check constraints are automatically disabled for direct path loads. Conventional path shouldn't have this problem.
There's a REENABLE clause to enable the constraints at the end of a direct path load.
The constraint can't "automatically" disable - check it's current state by issuing this SQL (as the table/constraint owner):
select status, validated from user_constraints where constraint_name = 'theNameOfYourConstraint'
It might be the case that it has been disabled prior to your operation. It will remain in that state until explicitly enabled.
[Additional]
I see from a later comment now that the disabling appears to be due to a direct path load in SQL*Loader. If you want to prevent this I think your only option is to use a conventional load and process the exceptions. Binding the file to an external table definition will allow you to use a more procedural approach since you can process the file as if it were an Oracle table.
From the Oracle docs:
Integrity constraints that depend on other rows or tables, such as referential constraints, are disabled before the direct path load and must be reenabled afterwards.
You'll have to provide us with some more information/code.
Describe your constraint.
How are you inserting records?
insert statement
calling a procedure
sql loader
etc.
Reproduce your problem:
select *
from user_constraints
where constraint_name = 'theNameOfYourConstraint';
do your insert here
select *
from user_constraints
where constraint_name = 'theNameOfYourConstraint';
and show us the output.