I need to create a sequence in the database that cannot be using sequence or identity.
There is a table in the database called File where all the files that users send in different areas of the system are stored.
It contains the id (primary key), name, type, folder, number, hash...
CREATE TABLE dbo.[File]
(
FileId uniqueidentifier NOT NULL,
Name nvarchar(30) NOT NULL,
FileTypeId int NOT NULL,
FileFolderId int NOT NULL,
Number int NOT NULL,
Hash nvarchar(50) NOT NULL
...
) ON [PRIMARY]
And then for each feature there is a table expanding the properties of the File table, an example is ContractFile.
It has the same id of the File table and with a few more fields and the id of the Contract table, creating the relation.
CREATE TABLE dbo.ContractFile
(
FileId uniqueidentifier NOT NULL,
ContractId uniqueidentifier NOT NULL
...
) ON [PRIMARY]
So the filename should follow a pattern.
050#H4G5H4G244#001.pdf
050#H4G5H4G244#002.pdf
060#H4G5H4G244#001.pdf
The first 3 digits is a code that is in the FileType table.
The digits in the middle is the code in the Contract table.
And the last 3 is the sequence that was inserted.
Then it groups the string by the FileType and the Contract.
So I created a trigger in the ContractFile table for when inserting it get the biggest number for that FileType and for the Contract and add +1, setting the Number field of the File table.
Then the file name is updated (in the same trigger)
CREATE TRIGGER [dbo].[tgContractFileInsert]
ON [dbo].[ContractFile]
FOR INSERT
AS
BEGIN
SET NOCOUNT ON
UPDATE dbo.File
SET Number = COALESCE(
(SELECT MAX(AR.Number)
FROM dbo.ContractFile NOA
INNER JOIN dbo.File AR
ON AR.FileId = NOA.FileId
WHERE NOA.ContractId = I.ContractId AND
AR.FileTypeId = T.FileTypeId
),
0) + 1
FROM dbo.File T WITH (XLOCK)
INNER JOIN Inserted I
ON I.FileId = T.FileId
WHERE T.Number IS NULL
UPDATE dbo.File
SET Name = dbo.fnFileName(AP.Code, NOB.Code, T.Numero, T.Name)
FROM dbo.File T
INNER JOIN Inserted I
ON I.FileId = T.FileId
INNER JOIN dbo.FileType AP
ON AP.FileTypeId = T.FileTypeId
INNER JOIN dbo.Contract NOB
ON NOB.ContractId = I.ContractId
END
At first it works, but when we have a large volume being inserted, there is a deadlock.
And from what I'm seeing also when inserting more than one record will end up getting the same number, since the Inserted table will bring two records and the +1 is not checking this.
How could I solve this? What is the best way?
Avoid deadlock, will the sequence be correct even inserting more than one record at a time and have a good performance?
Related
I have an excel file with thousand of rows which I need to use to delete/update/insert some tables.
The excel provides the following data: provider_id, country_name, locale, property1, property2.
The tables which need to be updated are:
provider_country with columns : provider_country_id, provider_id, country_id, property1, property2 and
provider_country_language with columns : provider_country_language_id, provider_country_id, language_id.
I can also use table country with columns (for joins): country_id, country_name.
And table language with columns: language_id, locale, country_id.
The fields which need to be updated are country_id, language_id, property1,property2 (from provider_country and provider_country_language)
I have created a temporary table with all the data from the excel:
CREATE TABLE #TempProviderCountryLanguage(
[provider_id] int NULL,
[country_name] nvarchar(50) NULL,
[locale] nvarchar(10) NULL,
[property1] int NULL,
[property2] decimal(5,2) NULL
)
INSERT INTO #TempProviderCountryLanguage VALUES
(1,N'Provider1',N'Brazil',N'en-br',4,NULL)
INSERT INTO #TempProviderCountryLanguage VALUES
(1,N'Provider1',N'Brazil',N'pt-br',4,NULL)
INSERT INTO #TempProviderCountryLanguage VALUES
(1,N'Provider1',N'Denmark',N'da-dk',4,12.21)
INSERT INTO #TempProviderCountryLanguage VALUES
(2,N'Provider2',N'Denmark',N'da-dk',5,14.21)
......
MERGE [provider_country] AS TARGET
USING (
SELECT tb.provider_id
,c.country_id
,l.language_id
,tb.property1
,tb.property2
FROM #TempProviderCountryLanguage tb
INNER JOIN country c ON c.country_name = tb.country_name
INNER JOIN language l ON l.locale = l.locale
) AS SOURCE
ON (
TARGET.provider_id = SOURCE.provider_id AND
TARGET.country_id = SOURCE.country_id
)
WHEN MATCHED
THEN
UPDATE
SET TARGET.country_id = SOURCE.country_id,
TARGET.property1 = SOURCE.property1,
TARGET.property2 = SOURCE.property2
WHEN NOT MATCHED BY TARGET
THEN
INSERT (
provider_id
,country_id
,property1
,property2
)
VALUES (
SOURCE.provider_id
,SOURCE.country_id
,SOURCE.property1
,SOURCE.property2
)
WHEN NOT MATCHED BY SOURCE THEN
DELETE;
For the provider_country_language I plan to make another merge.
I am trying to update the tables using merge but I have a problem because I cannot make a unique select here (somehow I would need the language_id as well):
ON (
TARGET.provider_id = SOURCE.provider_id AND
TARGET.country_id = SOURCE.country_id
)
And the error is :
The MERGE statement attempted to UPDATE or DELETE the same row more
than once. This happens when a target row matches more than one source
row. A MERGE statement cannot UPDATE/DELETE the same row of the target
table multiple times. Refine the ON clause to ensure a target row
matches at most one source row, or use the GROUP BY clause to group
the source rows.
How can I make this work and make sure all the tables are updated correctly?(not necessarily using Merge)
And from performance point of view what would be the best approach? (only the INSERT INTO will be performed thousand of times...)
I would like to create a trigger on update. There are two tables involved, cylinder and jobs. When the statusId in the cylinder changes to a certain value say 2 it changes a flag field in both tables. The cylinder table is update quite frequently. I believe I need to check the inserted and deleted tables it see if the statusId field changes to my value of 2.
Here are simple versions of my tables. They are related thru jobNumb
CREATE TABLE Cylinder (
cylinderId int IDENTITY(1,1) PRIMARY KEY,
jobNumb int NOT NULL,
statusId int,
flag int
)
CREATE TABLE Jobs (
jobId int IDENTITY(1,1) PRIMARY KEY,
jobNumb int NOT NULL,
statusId int,
reShip int,
flag int
)
Well, you need 2 different update statements since an update statement can only ever update a single table.
However, if the flag in both tables should always be the same value for related rows, you can achieve this by adding a foreign key constraint with on update cascade on this column.
A trigger for update would look something like this:
CREATE TRIGGER ON Cylinder
FOR UPDATE
AS
-- updates jobs flag
UPDATE j
SET flag = CASE WHEN i.statusId = 2 THEN 1 ELSE 0 END -- or whatever values suited for your flag
FROM inserted i
INNER JOIN deleted d ON(i.cylinderId = d.cylinderId AND i.statusId <> d.statusId)
INNER JOIN Jobs j ON(i.jobNumb = j.jobNumb)
-- updates cyliner flag
UPDATE c
SET flag = CASE WHEN i.statusId = 2 THEN 1 ELSE 0 END -- or whatever values suited for your flag
FROM inserted i
INNER JOIN deleted d ON(i.cylinderId = d.cylinderId AND i.statusId <> d.statusId)
INNER JOIN Cylinder c ON(i.jobNumb = c.jobNumb)
GO
I have two tables which are heavily queried by multiple users. Average 100+ (update/select) queries/second requests are made for these tables.
Parent
Child
*GrantParent is not involved in join so, I said only two tables
I need to reorder all children for each parent. There can be 3000-4000 parents and each parent may have around same number of children.
Column Types:
ParentID GUID
ChildIndex int
FileID Varchar
IsDeleted bit
Tables have clustered index on PK and non-clustered index on columns being used in where.
UPDATE C SET C.ChildIndex = T.ReOrderedChildIndex FROM [Child] C INNER JOIN
(
SELECT ROW_NUMBER() OVER (PARTITION BY dbo.Child.[ParentID] ORDER BY [ChildIndex] asc) AS ReOrderedChildIndex,
dbo.Child.ChildIndex,
dbo.Child.FileID,
dbo.Child.ParentID
FROM dbo.Child WITH (NOLOCK) INNER JOIN
dbo.Parent WITH (NOLOCK) ON dbo.Child.ParentID = dbo.Parent.ParentID
WHERE (dbo.Parent.GrandParentID = 1) AND (dbo.Child.IsDeleted = 0)
) T
ON C.FileID =T.FileID AND (C.ParentID=T.ParentID) AND (C.IsDeleted = 0)
It looks above query take longer time and put select queries on wait even I have used WITH (NOLOCK) in all data selection stored procedures.
There is another query which reorder parents in same way as done for childs in above query.
In Activity Monitor the locks are shown for select stored procedures.
What is the best way to reorder perform reordering?
I am having following issues and believe they are stems from these queries:
1- Randomly deadlock occur.
2- Often connection pool time out occurs.
*Database is accessed by a windows application using Entlib 4.0 with connection pooling enabled, pool max size 200.
SQL Server 2008 R2
I'd recommend restructuring your data to a more flexible schema. This schema will allow multiple levels so you can merge GrandParent, Parent, and Child into one logical relationship table and one logical details table. You'll also be able to take advantage of indexes to reduce locks and improve performance.
You'll have to re-build your hierarchy after any relationship changes. The way I wrote the script below should minimize this impact on your system. You will no longer be updating the entire table, just the pieces that have changed.
Schema:
CREATE TABLE dbo.EntityName
(
ID INT IDENTITY(1,1),
ParentID INT -- Todo: Add foreign key back to dbo.EntityName
-- Todo: Add primary key
);
GO
CREATE TABLE dbo.Hierarchy
(
ParentID INT, -- Todo: Add foreign key back to dbo.EntityName
ChildID INT, -- Todo: Add foreign key back to dbo.EntityName
ChildLevel INT
);
GO
Populate script (slightly rough around the edges):
CREATE PROCEDURE [dbo].[uspBuildHierarchy]
AS
BEGIN
SET NOCOUNT ON;
CREATE TABLE #Hierarchy
(
ParentID INT,
ChildID INT,
ChildLevel INT
);
-- Add the root of your hierarchy
INSERT INTO #Hierarchy VALUES (1, 1, 0);
DECLARE #ChildLevel INT = 1,
#LastCount INT = 1;
WHILE (#LastCount > 0)
BEGIN
INSERT INTO #Hierarchy
SELECT
E.ParentID,
E.ID,
#ChildLevel + 1
FROM dbo.EntityName E
INNER JOIN #Hierarchy H ON H.ChildID = E.ParentID
AND H.ChildLevel = (#ChildLevel - 1)
LEFT JOIN #Hierarchy EH ON EH.ParentID = E.ParentID
AND EH.ChildID = E.ID
WHERE EH.ChildLevel IS NULL;
SET #LastCount = ##ROWCOUNT;
SET #ChildLevel = #ChildLevel + 1;
END
MERGE INTO dbo.Hierarchy OH
USING
(
SELECT
ParentID,
ChildID,
ChildLevel
FROM #Hierarchy
) NH
ON OH.ParentID = NH.ParentID
AND OH.ChildID = NH.ChildID
WHEN MATCHED AND OH.ChildLevel <> NH.ChildLevel THEN
UPDATE
SET ChildLevel = NH.ChildLevel
WHEN NOT MATCHED THEN
INSERT
VALUES
(
NH.ParentID,
NH.ChildID,
NH.ChildLevel
)
WHEN NOT MATCHED BY SOURCE
THEN DELETE;
END
GO
Query for all of an entity's children:
SELECT *
FROM dbo.EntityName E
INNER JOIN dbo.Hierarchy H ON H.ChildID = E.ID
AND H.ParentID = #EntityNameID;
I want to write a trigger to the view, VW_BANKBRANCH:
If the inserted row contains a bankcode that exists in the table, then update the
bName column of bank table with the inserted data
If not, insert rows to bank table to reflect the new information.
But my trigger is not working..
My tables
CREATE TABLE bank(
code VARCHAR(30) PRIMARY KEY,
bName VARCHAR(50)
);
CREATE TABLE branch(
brNum INT PRIMARY KEY,
brName VARCHAR(50),
braddress VARCHAR(50),
bcode VARCHAR(30) REFERENCES bank(code)
);
CREATE VIEW VW_BANKBRANCH
AS
SELECT code,bname,brnum,brName
FROM bank ,branch
WHERE code=bcode
My trigger
CREATE TRIGGER tr_VW_BANKBRANCH_INSERT ON VW_BANKBRANCH
INSTEAD OF INSERT
AS
BEGIN
DECLARE #insertedBankCode INT
#insertedbname varchar
#insertedbrnum int
#insertedbrName varchar
SELECT #insertedBankCode = code
FROM INSERTED
IF(#insertedBankCode=code)
SET code=#insertedBankCode
bname=#insertedbname
brnum=#insertedbrnum
brName=#insertedbrName
ELSE
insert(code,bname,brnum,brName)
END
I've adapted the instead of trigger on the view below - I'm assuming you want to upsert both bank and branch accordingly (although note that the branch address is not currently in the view).
That said, I would be careful of (ab)using an instead of trigger on an INSERT to do upserts - this might not be entirely intuitive to the reader.
Also, remember that the INSERTED pseudo table could contain a SET of rows, so needs to be adjusted to set based approach accordingly.
CREATE TRIGGER tr_VW_BANKBRANCH_INSERT ON VW_BANKBRANCH
INSTEAD OF INSERT
AS
BEGIN
SET NOCOUNT ON;
UPDATE b
SET bname = i.bname
FROM bank b
INNER JOIN inserted i
ON i.code = b.code;
UPDATE br
SET
br.brName = i.brName,
br.braddress = NULL -- TODO add this to the view
FROM branch br
INNER JOIN inserted i
ON br.bcode = i.code
AND br.brNum = i.brNum;
INSERT INTO bank(code, bname)
SELECT code, bname
FROM inserted i
WHERE NOT EXISTS
(SELECT 1 FROM bank b WHERE b.code = i.Code);
INSERT INTO Branch(brNum, brName, braddress, bcode)
SELECT brNum, brName, NULL, code
FROM inserted i
WHERE NOT EXISTS
(SELECT 1
FROM branch br
WHERE br.bcode = i.Code AND br.brNum = i.brNum);
END;
GO
SqlFiddle here - I've also adjusted the view to use a JOIN, rather than the old style of WHERE joins.
If you have SqlServer 2008 or later, you could also use MERGE instead of separate inserts and updates.
We have a table where we store all the exceptions (message, stackTrace, etc..), the table is getting big and we would like to reduce it.
There are plenty of repeated StackTraces, Messages, etc, but enabling compression produces a modest size reduction (10%) while I think much bigger benefits could come if somehow Sql Server will intern the strings in some per-column hash-table.
I could get some of the benefits if I normalize the table and extract StackTraces to another one, but exception messages, exception types, etc.. are also repeated.
Is there a way to enable string interning for some column in Sql Server?
There is no built-in way to do this. You could easily do something like:
SELECT MessageID = IDENTITY(INT, 1, 1), Message
INTO dbo.Messages
FROM dbo.HugeTable GROUP BY Message;
ALTER TABLE dbo.HugeTable ADD MessageID INT;
UPDATE h
SET h.MessageID = m.MessageID
FROM dbo.HugeTable AS h
INNER JOIN dbo.Messages AS m
ON h.Message = m.Message;
ALTER TABLE dbo.HugeTable DROP COLUMN Message;
Now you'll need to do a few things:
Change your logging procedure to perform an upsert to the Messages table
Add proper indexes to the messages table (wasn't sure of Message data type) and PK
Add FK to MessageID column
Rebuild indexes on HugeTable to reclaim space
Do this in a test environment first!
Aaron's posting answers the questions of adding interning to a table, but afterwards you will need to modify your application code and stored-procedures to work with the new schema.
...or so you might think. You can actually create a VIEW that returns data matching the old schema, and you can also support INSERT operations on the view too, which are translated into child operations on the Messages and HugeTable tables. For readability I'll use the names InternedStrings and ExceptionLogs for the tables.
So if the old table was this:
CREATE TABLE ExceptionLogs (
LogId int IDENTITY(1,1) NOT NULL PRIMARY KEY,
Message nvarchar(1024) NOT NULL,
ExceptionType nvarchar(512) NOT NULL,
StackTrace nvarchar(4096) NOT NULL
)
And the new tables are:
CREATE TABLE InternedStrings (
StringId int IDENTITY(1,1) NOT NULL PRIMARY KEY,
Value nvarchar(max) NOT NULL
)
CREATE TABLE ExceptionLogs2 ( -- note the new name
LogId int IDENTITY(1,1) NOT NULL PRIMARY KEY,
Message int NOT NULL,
ExceptionType int NOT NULL,
StackTrace int NOT NULL
)
Add an index to InternedStrings to make the value lookups faster:
CREATE UNIQUE NONCLUSTERED INDEX IX_U_InternedStrings_Value ON InternedStrings ( Value ASC )
Then you would also have a VIEW:
CREATE VIEW ExeptionLogs AS
SELECT
LogId,
MessageStrings .Value AS Message,
ExceptionTypeStrings.Value AS ExceptionType,
StackTraceStrings .Value AS StackTrace
FROM
ExceptionLogs2
INNER JOIN InternedStrings AS MessageStrings ON
MessageStrings.StringId = ExceptionLogs2.Message
INNER JOIN InternedStrings AS ExceptionTypeStrings ON
ExceptionTypeStrings.StringId = ExceptionLogs2.ExceptionType
INNER JOIN InternedStrings AS StackTraceStrings ON
StackTraceStrings.StringId = ExceptionLogs2.StackTrace
And to handle INSERT operations from unmodified clients:
CREATE TRIGGER ExceptionLogsInsertHandler
ON ExceptionLogs INSTEAD OF INSERT AS
DECLARE #messageId int = SELECT StringId FROM InternedStrings WHERE Value = inserted.Message
IF #messageId IS NULL
BEGIN
INSERT INTO InternedStrings ( Text ) VALUES ( inserted.Message )
SET #messageId = SCOPE_IDENTITY()
END
DECLARE #exceptionTypeId int = SELECT StringId FROM InternedStrings WHERE Value = inserted.ExceptionType
IF #exceptionTypeId IS NULL
BEGIN
INSERT INTO InternedStrings ( Text ) VALUES ( inserted.ExceptionType )
SET #exceptionTypeId = SCOPE_IDENTITY()
END
DECLARE #stackTraceId int = SELECT StringId FROM InternedStrings WHERE Value = inserted.StackTrace
IF #stackTraceId IS NULL
BEGIN
INSERT INTO InternedStrings ( Text ) VALUES ( inserted.StackTrace )
SET #stackTraceId = SCOPE_IDENTITY()
END
INSERT INTO ExceptionLogs2 ( Message, ExceptionType, StackTrace )
VALUES ( #messageId, #exceptionTypeId, #stackTraceId )
Note this TRIGGER can be improved: it only supports single-row insertions, and is not entirely concurrency-safe, though because previous data won't be mutated it means that there's a slight risk of data duplication in the InternedStrings table - and because of a UNIQUE index the insert will fail. There are different possible ways to handle this, such as using a TRANSACTION and changing the queries to use holdlock and updlock.