IN CDC, Is there any way by which, we can get information who(user) made changes in table. and by which system (computer name)?
No (see this post and this post).
You’d need a trigger, and then have to correlate changed rows between CDC and wherever your trigger logs actions. You could add a column to the table to make that easier, but then you’d only know who made the last change.
I think it would be a better approach to either add SQL Audit (though I don’t recall how good the functionality was 15 years ago, and whether what you need is supported by your edition) instead of a trigger, or force data changes through a stored procedure, where you can log as part of the action (eg using the OUTPUT clause).
Related
I am researching archiving options for our database application (highly normalized schema) and would appreciate expert feedback. We are using Sql Server 2005, but if something works only in 2008 R2 that may be an option for us.
Primary reason for archiving is to remove old data on an annual basis. The criteria to determine which objects can be archived will not be straightforward (ie: not just filtering by a date, but many more considerations involved).
Archiving needs to be basically a push button on the application (ie: not by actual DBA on the database server).
Data should be retrievable, but perhaps by special request. Perhaps an object and all its related pieces could be searched for and brought back into the current database? (Again, via the application interface.)
Another important requirement is to maintain integrity of related data. If an archived object is related to a non-archived object, I want to ensure the non-archived object can't be deleted through the interface. Currently we have many checks in place to ensure you can't delete items if they're in use, and I hesitate to alter all of those checks to join an _archive table or use a new view. Is there another way?
I have read about table/index partitioning and although it is interesting, it sounds like perhaps a LOT of work considering how many stored procedures, views, indexes, etc that we use.
What is your motivation for archiving?
You mention you want to "remove old data" but since you need it to be constantly available that doesn't make any sense.
The easiest thing to do in your situation will be a "soft" archive, where you add an Archived bit field to all your tables that indicates if a row is active or not. Then all your existing referential checks stay in place, but you need to add a filter on that bit in your views or queries, and add it to most of your indexes.
You don't really need to do an offload since you can't move the data off the server anyways.
I have a profile page (less than 10k users) and I need to track every change and update that is made through out the application, by the users. Whether a user updates the profile picture or adds an extra space in a "comments" field, I need to store the previous data.
In short: I need to store everything.
Is there some sort of "tracking-history-and-changes-in-sql-server" software or do I have to implement it myself?
If I have to implement it myself I can think of three ways to do it:
I keep everything in the same table
but I add a key column that
specifies which row is active
and which is old.
I add a new table called history
where I store the column name that had the
change, when it was changed and what
the old data was.
I add a history table for each table
in the database. It looks the same
but only keep track of each tables
history.
Has anyone had a similar problem and how did you solve it?
This was built using mvc 4 and it's a
normal website.
EDIT
I'm mostly interested in existing solutions/software, but If there are none I would have to do it myself.
Has anyone used SQL Data Compare?
Where I worked last everything had to be logged fully. (working with goverment organisations). We never updated or deleted data.
What you would do is have a start date and an end date on each row. To do an update you would update the old data to have an enddate then insert a new row in the table. To do a delete you would put an enddate on the row with a null enddate. We also had an "updated by" column to put the userid
I used the third approach to do that but didn't create a history table for all tables of my DB but history tables for most important tables. You can use triggers to do that , create trigger for Update. You can read more about Triggers here and here
Microsoft offers Change Tracking and Change Data Capture for awhile now. These technically offer the tracking of all your changes in your database, which suits your purpose. Just note that CT is available in most versions, whereas CDC used to be only available in Enterprise until SQL Server 2016, where they made it available for Standard too.
ApexSQL Log does pretty much the same as well, but if you're using SQL Server then it's integrated in your software already.
have you considered enabling full transaction logging on your database and then using some of the log reading tools to monitor data changes. ApexSQL Log is by far the best log reader on the market but there are other solutions out there. SQL Log Rescue from Red Gate is free but it's only for sql server 2000.
Using this approach you dont need to make any other changes in your database or in your application since every transaction is automatically logged when database is in full recovery mode.
Hope this helps.
Is there any handy tool that can make updating tables easier? Usually I got an Excel file with the original value in one column and new value in another column. Then I write a formula in Excel to create the 'update' statement. Is there any way to simplify the updating task?
I believe the approach in SQL server 2000 and 2005 would be different, so could we discuss them both? Thanks.
In addition, these updates usually request by "non-programmer" (which means they don't understand SQL, so it may not feasible to let them do query), is there any tool that can let them update the table directly without having DBAs do this task? Also, that tool needs to limit the privilege to only modify certain tables. And better has a way rollback the change.
Create a DTS package that will import a csv file, make the updates and then archives the file. The user can drop the file in a specific folder designated for the task or this can be done by an ops person. Schedule the DTS to run every hour, day, etc.
In case your users would insist that they keep using Excel, you've got several different possibilities of getting the data transferred to SQL Server. My preferred one would be to use DTS/SSIS, as mentioned by buckbova.
However, another method is by using OPENROWSET(), which makes it possible to query your Excel file as if it was a table. I wrote a small article about it here: http://blog.hoegaerden.be/2010/03/29/retrieving-data-from-excel/
Another approach that hasn't been mentioned yet (I'm not a big fan of letting regular users edit data directly in the DB), any possibility of creating a small custom application for them?
There you go, a couple more possible solutions :-)
Valentino.
I think the best approach is to expose a view on your data accessible to users who are allowed to do updates, and set up triggers on the view to perform the actual updates on the underlying data. Restrict change to only the columns they should be changing.
This technique can work on SQL Server 2000 and 2005.
I would add audit triggers on the underlying tables so you can always track changes.
You'll have complete control, and they can connect to it with Access or whatever and perform their maintenance.
You could create some accounts in SQL Server for these users and limit their access to only certain tables and columns along with onlu select / update / insert privileges. Then you could create an access database with linked tables to these.
I need to create in my DB a log, that every action in the program should be written there.
I will also want to store additional data to it for example have the table and row the action was applied to.
In other words I want the log to be dynamic and should be able to refer to the other tables in the database.
The problem is, I don't know how to relate all the tables to this log.
Any ideas?
You have two choices here:
1) modify your program to add logging for every db access
2) add triggers to each table in your db to perform logging operations.
I don't recommend one logging table for all tables. You will have locking issues if you do that (every insert, update and delete in every table woudl have to hit this one, bad idea). Create a table for each table that you want to audit. There are lots of possible designs for the table, but they usually include some variant of old vlaue, new value, date changed, and user who did the change.
Then create triggers on each table to log the changes.
I know SQL Server 2008 also has a systemic way to set up auditing, this would be easier to set up than manual auditing and might be enough to lure your company into using 2008.
I have a problem with a Database at my work. There is currently auditing in place, but its clunky, requires a lot of maintence, and it falls short in a few regards. So I am replacing it.
I want to do this in as generic of a way as possible and have designed the tables, and how everything will link and be updated.
Now, thats all fine and good, but I want to be able to write a generic way to insert records into these audit tables. (Without having to enter a command for each column in each table being changed.)
Is there anyway within a Stored Procedure to iterate over all the columns in a table? And I would like to write this in such a way that it will work with several tables, and automatically pickup and audit added columns and such.
Any ideas?
EDIT: Guess I should Clarify. I will be auditing data that is in the tables. But I will be using the same table(s) to store the audited data for every table in the database.
And I can not use Triggers because usually, when an update occurs, it occurs across multiple tables, but I would like all of these updates to be part of a single Change Set.
That is not a problem, because I can do all the Updates from within a single Stored Proc. I would just prefer some way like a loop, that i can get all the updated fields, figure out which ones changed, and the insert those changed ones into the audit table.
And I would like to do this without have a long list of if statements and insert statements for each column. (By doing this in a generic loop, it will handle added columns automatically and not be bothered by deleted columns)
By "added columns" I guess you are looking to audit DDL. If you use SQL 2005, then you want this link.
If don't use SQL 2005, then you probably want to either use one of the many SQL schema comparison tools, like SQL Red Gates tool set probably has something in there.
If you don't have $ for tools, then you might just want to run periodic queries against information_schema.tables and information_schema.columns. By periodically capturing these in permenant tables, you can identify when they have gained or lost rows (and hence a schema changed occured)
If you are doing data audit instead, then you'll want want to code generate some triggers, again using information_schema.tables and information_schema.columns.
There would be performance considerations, but you could add insert and update triggers to all of your tables, and have the triggers insert into your audit tables.
Use DDL Triggers (assuming you have SQL Server 2005+)!
http://www.sqlteam.com/article/using-ddl-triggers-in-sql-server-2005-to-capture-schema-changes
http://technet.microsoft.com/en-us/library/ms189871.aspx
That could be done if you were using a data access layer that could trap which tables and columns are being update and generating the insert statements for the audit table. In a stored procedure? Which stored procedure? Do you have a single one that does updates? Or are you creating one per table?
If it's an option for you, just upgrade to sql server 2008 and turn on Change Data Capture.