I'm working on a simple VB NET/WPF application that communicates with a SQL Server 2012 database, which at least two or more people could be using at the same time.
To keep track of any and every change in the database (INSERT/UPDATE/DELETE), I'm implementing a simple trigger-based auditing system, where obviously, any data getting updated/deleted/inserted is saved in its corresponding auditing table, while also keeping track of the user and the date/time of the operation.
I obviously found tons of guides while searching. But I noticed that most of these SQL Server guides would use JOINs in their queries to extract the data stored in the Inserted or Deleted tables, while I achieved the same result without any JOINs.
Exemple :
My INSERT trigger :
CREATE TRIGGER [HumanResources].[after_insert_humanresources_shift]
ON [HumanResources].[Shift]
AFTER INSERT
AS
BEGIN
INSERT INTO [HumanResources].[Shift_Audit]
(
-- [EventID], [EventBy] and [EventOn] have autovalues
[EventType],
[ShiftID],
[Name],
[StartTime],
[EndTime],
[ModifiedDate]
)
SELECT
'INSERT',
[ShiftID],
[Name],
[StartTime],
[EndTime],
[ModifiedDate]
FROM
[Inserted]
END
This guide's INSERT Trigger :
create trigger tblTriggerAuditRecord on tblOrders
after insert
as
begin
insert into tblOrdersAudit
(OrderID, OrderApprovalDateTime, OrderStatus, UpdatedBy, UpdatedOn )
select i.OrderID, i.OrderApprovalDateTime, i.OrderStatus, SUSER_SNAME(), getdate()
from tblOrders t
inner join inserted i on t.OrderID=i.OrderID
end
My DELETE and UPDATE triggers do exactly the same thing. And all these work for both single D/U/I queries or multiple ones (multiple Updates in a single query for instance). Is there a specific reason why I should be using JOINs ?
Yes the "inner join inserted i on t.OrderID=i.OrderID" checks for dependencies on orders table, and will insert only records where there are genuine orders in the orders table, inserting to independent tables does not test the data for dependencies at entity level. hope that helps.
Related
Disclaimer: I'm not a SQL expert. I'm trying to insert records into a child table before inserting them into the parent table. (After saying that I'm starting to wonder if this is even a good idea.) The parent table record holds a reference to the child table record, and said-reference cannot be null. This necessitates me inserting into the child table first, then linking to the parent table during a secondary insert.
Anyway for some reason, this code randomly produces orphaned records in the IdentifyingData (child) table e.g., they have no entry in the FraudScore (parent) table even though they should.
Here's why I'm confused. In trying to resolve this issue, I started dumping the contents of the #tempFraudScore table into a physical audit table so I can see exactly what's going on during data transformation. When I switch the below code that inserts into FraudScore from #tempFraudScore to insert from the audit table, all the child records successfully get a parent record created. This makes no sense to me.
insert into IdentifyingData (EntryDateTime, IdentifyingDataTypeId, Value, Source)
select distinct GETDATE(), tfs.IdentifyingDataTypeId, tfs.Value, 'SSIS'
from #tempFraudScore tfs
where not exists (
select id.IdentifyingDataTypeId, id.Value
from IdentifyingData id
where tfs.IdentifyingDataTypeId = id.IdentifyingDataTypeId
and tfs.Value = id.Value
);
update tfs
set tfs.IdentifyingDataId = id.Id
from #tempFraudScore tfs
inner join IdentifyingData id on
tfs.Value = id.Value and
tfs.IdentifyingDataTypeId = id.IdentifyingDataTypeId;
insert into FraudScore (EntryDateTime, FraudCriteriaId, AccountId, IdentifyingDataId, Score, Source)
select distinct
GETDATE() EntryDateTime,
tfs.FraudCriteriaId,
tfs.AccountId,
tfs.IdentifyingDataId,
tfs.Score,
'SSIS'
from #tempFraudScore tfs
inner join FraudCriteria fc on
tfs.FraudCriteriaId = fc.Id
and fc.UniqueEntryPeriod = 0
where not exists (
select fs.AccountId, fs.FraudCriteriaId, fs.IdentifyingDataId
from FraudScore fs
where tfs.AccountId = fs.AccountId
and tfs.FraudCriteriaId = fs.FraudCriteriaId
and tfs.IdentifyingDataId = fs.IdentifyingDataId
);
#tempFraudScore comes pre-populated with all of the necessary fields except for IdentifyingDataId; that has to be created by first inserting into IdentifyingData, then updating the variable table with the created ID. Below is the structure of the variable table:
declare #tempFraudScore table(
FraudCriteriaId int,
AccountId bigint,
IdentifyingDataId bigint,
IdentifyingDataTypeId smallint,
Value varchar(100),
Score int
);
Could someone please tell me what could be causing these orphaned IdentifyingData records? Should I reconsider how the relationships between these two tables are structured? I'm trying to make things so that once a certain IdentifyingData record is put into the system, it won't get duplicated; it'll simply be referenced by newly created FraudScore records.
Edit
Attached is a screenshot from the audit table that shows the progress of data transformation for a single value (the Value column is the same value for these records; I'm blurring it out for privacy's sake). Note that despite the message "Post-FraudScore Insert", the record in question was never actually inserted into the FraudScore table.
Edit2 (2/6/2018): I've added the following code to the stored procedure in trying to troubleshoot this issue. I had a value (99999) that appeared in the _Audit table's Value column, but not the second table's Value column despite the code simply dumping all data into these two tables from the same source! I'm not sure if it matters, but this stored procedure gets kicked off from an SSIS package's Execute SQL Task with an IsolationLevel of "Serializable". That-said, I'm not explicitly using transactions anywhere in the code, and the TransactionOption for that Execute SQL Task is set as "Supported". I have no clue if this would have anything to do with the issue.
insert into FraudScoreIdentifyingData_Audit
select 'Post-IdentifyingData Update', GETDATE(), FraudCriteriaId, AccountId, IdentifyingDataId, IdentifyingDataTypeId, Value, Score
from #tempFraudScore;
insert into FraudScoreIdentifyingData
select GETDATE(), FraudCriteriaId, AccountId, IdentifyingDataId, IdentifyingDataTypeId, Value, Score, 1
from #tempFraudScore;
Here are the two tables' schemas:
Can't say what causing the problem.
Parent Table=FraudScore
Child Table=IdentifyingData
how they are related ?First you insert record in FraudScore then using output clause if you have more than one insert,insert record in IdentifyingData
But this is ideal situation to use OUTPUT clause even if problem do not solve because of this.
--data type similar to IdentifyingData
declare #tbl table(Id int,Value int,IdentifyingDataTypeId int)
declare #CurrentDateTime datetime=GETDATE()
begin try
begin transaction
insert into IdentifyingData (EntryDateTime, IdentifyingDataTypeId
, Value, Source)
OUTPUT INSERTED.Id, INSERTED.Value, INSERTED.IdentifyingDataTypeId
INTO #tbl
select distinct #CurrentDateTime, tfs.IdentifyingDataTypeId
, tfs.Value, 'SSIS'
from #tempFraudScore tfs
where not exists (
select id.IdentifyingDataTypeId, id.Value
from IdentifyingData id
where tfs.IdentifyingDataTypeId = id.IdentifyingDataTypeId
and tfs.Value = id.Value
);
update tfs
set tfs.IdentifyingDataId = id.Id
from #tempFraudScore tfs
inner join #tbl id on
tfs.Value = id.Value and
tfs.IdentifyingDataTypeId = id.IdentifyingDataTypeId;
insert into FraudScore (EntryDateTime, FraudCriteriaId, AccountId,
IdentifyingDataId, Score, Source)
select distinct
#CurrentDateTime EntryDateTime,
tfs.FraudCriteriaId,
tfs.AccountId,
tfs.IdentifyingDataId,
tfs.Score,
'SSIS'
from #tempFraudScore tfs
inner join FraudCriteria fc on
tfs.FraudCriteriaId = fc.Id
and fc.UniqueEntryPeriod = 0
where not exists (
select fs.AccountId, fs.FraudCriteriaId, fs.IdentifyingDataId
from FraudScore fs
where tfs.AccountId = fs.AccountId
and tfs.FraudCriteriaId = fs.FraudCriteriaId
and tfs.IdentifyingDataId = fs.IdentifyingDataId
);
COMMIT
end TRY
begin CATCH
if(##trancount>0)
ROLLBACK
end CATCH
Turns out there was a single delete statement buried in one of my large stored procedures that was written incorrectly that was causing the problem.
In searching for the cause of this issue, I also had a DBA sit with me and he identified a part of my SSIS process that was reorganizing indexes; but it was doing so as the package continued to run and populate all the necessary underlying tables (including the one with the orphaned records). According to him, reorganizing or rebuilding indexes on tables while simultaneously attempting to add or remove records to those tables could also cause this problem; although in my specific case it was the incorrectly written, single delete statement.
I have a table in my Datamart; users are supposed to only read from that table… I checked and the count of the table went down, meaning, someone, a SP or a job, deleted records, and I sense it is not the first time.
My question: what is the less invasive, and simpler way of tracking this: I do not want to prevent this; I want to get the person’s name, or the SP/Job name, and the exact time it happened.
I am using: Microsoft SQL Server 2012 (SP1) - 11.0.3000.0 (X64): Business Intelligence Edition (64-bit) on Windows NT 6.2 (Build 9200: ) (Hypervisor)
I have ‘simple’ recovery model, I assume tracking it in the past is really challenging, so I am happy with retrieving this information in the future.
You can use a AFTER DELETE trigger on your table and log deleted values into a history log table with user information and deletion time, etc as follows
CREATE TRIGGER dbo.myTableDeleteTrigger
ON dbo.myTable
AFTER DELETE
AS
INSERT INTO myTableHistory (
-- columns from myTable
DeletedDate,
DeletedByUserId
)
SELECT
-- delete column values from Deleted temp table
GETDATE(),
USER_ID()
FROM Deleted
GO
I used a similar trigger to log data changes for insert, update and delete DMLs on a SQL database table as I explained in the referred tutorial
This below code will generate the Triggers for every table in your database.
Note: You can exclude the tables which you don't want the track in CTE
;WITH CTE AS(
SELECT TAB.name FROM SYS.objects TAB
where TAB.type='U'
)
SELECT '
GO
CREATE TRIGGER [dbo].[TRG_'+NAME+'_LOG] ON [dbo].['+NAME+']
FOR UPDATE,DELETE
AS
INSERT INTO LOG_'+NAME+'
(LOG_DTE,'+(SELECT STUFF((
SELECT ', '+ COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME=NAME FOR XML PATH('')),1,1,''))+')
SELECT getdate(),'+(SELECT STUFF((
SELECT ', '+ COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME=NAME FOR XML PATH('')),1,1,''))+'
from deleted
PRINT ''AFTER TRIGGER FIRED''
' FROM CTE order by name OFFSET 81 ROWS FETCH NEXT 571 ROWS ONLY
But before you need to create a table for every table with the name prefix Log_ of actual table.
Ex: If you have table Employee (Eid int, EName Varchar(250))
You need to have a table like
Log_Employee (LOG_EID int identity,Log_DTe Datetime, EID int FK, EName Varchar(250))
I have Costumers table and Costumers_update table. I have created a trigger that Cross Joins the fields from Costumers table to Costumers_update table. I'm trying to match the rows and columns properly but I get mismatches all over the place when I do a SELECT query from C# on my winform.
The intentions of the query is to have some kind of log from Costumers previous information to the current which is why I have the trigger.
CREATE TRIGGER TriggerModify
ON Costumers AFTER UPDATE AS
BEGIN
INSERT INTO Costumers_Update (
CostumerID,
ModifyDate,
Name_New,
Email_New,
Name_Old,
Email_Old,
--etc
)
SELECT [Old].ID,
GETDATE(),
[New].Name,
[New].Email,
[Old].Name,
[Old].Email,
FROM inserted AS [New]
CROSS JOIN
deleted AS [Old]
END
When I do a SELECT or SELECT DISTINCT I get everyhing mashed up.
The Costumers tables structur is almost the same as Costumers_update except the [OLD] fields.
I'm not sure what you mean by "mismatches all over the place".
However, you are doing a CROSS JOIN, so you are getting every combination of "new" and "old" values across multiple records.
I would suggest that you use a simple JOIN:
FROM inserted [New] JOIN
deleted [Old]
[New].CustomerId = [Old].CustomerId
I recently created a SQL trigger to replace a very expensive query I used to run to reduce the amount of updates my database does each day.
Before I preform an update I check to see how many updates have already occurred for the day, this used to be done by querying:
SELECT COUNT(*) FROM Movies WHERE DateAdded = Date.Now
Well my database has over 1 million records and this query is run about 1-2k a minute so you can see why I wanted to take a new approach for this.
So I created an audit table and setup a SQL Trigger to update this table when any INSERT or UPDATE happens on the Movie table. However I'm noticing the audit table is getting out of sync by a few hundred everyday (the audit table count is higher than the actual updates in the movie table). As this does not pose a huge issue I'm just curious what could be causing this or how to go about debugging it?
SQL Trigger:
ALTER TRIGGER [dbo].[trg_Audit]
ON [dbo].[Movies]
AFTER UPDATE, INSERT
AS
BEGIN
UPDATE Audit SET [count] = [count] + 1 WHERE [date] = CONVERT (date, GETDATE())
IF ##ROWCOUNT=0
INSERT INTO audit ([date], [count]) VALUES (GETDATE(), 1)
END
The above trigger only happens after an UPDATE or INSERT on the Movie table and tries to update the count + 1 in the Audit table and if it doesn't exists (IF ##ROWCOUNT=0) it then creates it. Any help would be much appreciated! Thanks.
Something like this should work:
create table dbo.Movies (
A int not null,
B int not null,
DateAdded datetime not null
)
go
create view dbo.audit
with schemabinding
as
select CONVERT(date,DateAdded) as dt,COUNT_BIG(*) as cnt
from dbo.Movies
group by CONVERT(date,DateAdded)
go
create unique clustered index IX_MovieCounts on dbo.audit (dt)
This is called an indexed view. The advantage is that SQL Server takes responsibility for maintaining the data stored in this view, and it's always right.
Unless you're on Enterprise/Developer edition, you'd query the audit view using the NOEXPAND hint:
SELECT * from audit with (noexpand)
This has the advantages that
a) You don't have to write the triggers yourself now (SQL Server does actually have something quite similar to triggers behind the scenes),
b) It can now cope with multi-row inserts, updates and deletes, and
c) You don't have to write the logic to cope with an update that changes the DateAdded value.
Rather than incrementing the count by 1 you should probably be incrementing it by the number of records that have changed e.g.
UPDATE Audit
SET [count] = [count] + (SELECT COUNT(*) FROM INSERTED)
WHERE [date] = CONVERT (date, GETDATE())
IF ##ROWCOUNT=0
INSERT INTO audit ([date], [count])
VALUES (GETDATE(), (SELECT COUNT(*) FROM INSERTED))
Hoping someone can give me a general example of how to do the following in MSSQL 2008/2005
I need to do the following in 1 stored procedure.
I need it to verify TableA has more than 1 record.
If TableA has more than one record then:
Delete all records from TableB AND copy the records from TableA to TableB
For the sake of argument and/or simplicity TableA and TableB Schemes are the same
This task wouldn't be that hard if I was performing the tasks in VB but I am trying to offload this work to the SQL server and I am not familiar on how to accomplish this.
Try something like this:
CREATE PROC DoStuff
AS
IF (SELECT COUNT(*) FROM TableA) > 1
BEGIN
DELETE TableB;
INSERT INTO TableB (ID, CustomerName)
SELECT ID, CustomerName
FROM TableA;
END
I would recommend looking at SQL Server integration services. It is designed to perform exactly the types of tasks you are trying to do.
Here are a couple of links to get you started:
http://sql-bi-dev.blogspot.com/2010/11/incremental-load-using-ssis-package.html
http://vsteamsystemcentral.com/cs21/blogs/applied_business_intelligence/archive/2007/05/21/ssis-design-pattern-incremental-loads.aspx
These go over using an Incremental Load which seems to be what you want to do.