I am working on developing an insurance app. We have designed a database table containing all user profile related information, as well as their approvals to different privacy policies.
My question is - which would be a better option to keep track of the history of all changes in the information described above?
Archive table or Audit Log table? or are both needed?
Details
Archive table - a copy of the current table will be created but will contain all the past data. All present data will be in the current table.
Audit Log table - generic table that contain columns that describe which tables and columns from those tables changed and to what values.
Is having one enough? or is it a standard practice to have both an archive table and an audit table to keep track of historical data of user profile related information?
Tried doing some research but all I could find was the archiving of the actual audit log data, not the archiving of the actual data. Makes it seem more confusing.
Related
It's my first time creating an audit log for a PoS WPF application and was wondering on how exactly do I implement an auditing system because it seems like each option available has its ups and downs. So far from reading numerous articles/threads, I've narrowed down a few common practices on audit logs:
1. Triggers - Unfortunately, due to the nature of my app, I can't make use of triggers as it has no way of knowing which user has done the action. So what I did instead was to create a Stored Procedure which will handle the customer insert along with the customer log insert and its details. The Customer_Id will be provided by the application when using the Stored Procedure.
2. Have an old and new value - My initial plan was to only include the latter since I can reference its old value with the new value from the row before it but storing the the old and new value seemed more sensible, complexity-wise.
3. Use a separate database for the log / 4. Foreign Keys - This is probably my main concern, if I decide to use a separate database for the audit table, then I couldn't setup foreign keys for the customer and employee involved.
I created a mock-up erd with a master-detail table result to be shown on the wpf app to display the log to an admin and would really like your thoughts on possible problems that may arise (There's also an employee table but I forgot to put it):
https://ibb.co/dheaNK
Here's a few info that might be of help:
The database will reside together with the wpf app, which is a single computer.
The amount of customers will be less than 1000.
The amount of regular employees will be 3.
The amount of admins will be 2.
You can enable CDC Change Data Capture on SQL Server database for a specific table
This will enable you to collect all data changes on the database table logged in special tables.
You can also refer to official documents too
Here is a list of DML commands and how data changes are logged in the CDC table created for the source database table
What is good about CDC is it comes default with SQL Server and you don't have to do anything for logging. The only requirement is SQL Server Agent should be running so that the changes can be reflected on log table.
I have the need to store the user information temporarily on SQL server to serve audit purpose. My idea is :
because SQL can only audit changed data, but I also want to store user's information who made the changes, so I've to figure somethings out to achieve this
when user log in to the application, create a local temporary table (#), name it after the user's current SPID , somethings like : #sp56, and store the information you need in there
when the audit triggers are fired, it will find the uses's nesscessary informations based on the SPID and fill the audit table with them
I also tried to audit the data in EF, but found it too complicated and not worthy to spend so much time in it
So my question is, is this possible using EF and SQL-Server? If there're any other better alternatives, I'm happy to hear!
We have Restaurant Inventory Control system that uses SQL Server 2008 R2.
It takes a very long time to add all the definition data: stock items, yields, packsizes, recipes, categories etc. So, our clients have asked if they can upload it from Excel.
Before I just jump in and start, I want to find out if there is a best practice way to do this.
I know all the tools: SSIS, stored procedures etc. But I'm looking for advice/resources that can help with the design process. How best to setup the spreadsheet, validate the data, create the child/parent relationships etc.
This must be a fairly common project -- so it must have a standard design/approach and that's what I'm looking for.
I think the design will depend on the technologies you're most comfortable with. If you're comfortable with SSIS and stored procedures, this is the general pattern I would use:
Excel Template - I wouldn't spend too much time on this, add the headers and sheets necessary for the tables. You can lock down certain things and/or implement rules, but most of your validation would be done in stored procs.
SSIS - Have a package that loads the excel data into Staging tables, have rows with errors get added to an error log to be presented to the user along with the validation issues from the stored procedures.
Staging Tables - Have one staging table per sheet/production table, have an ExecutionId column in each staging table to allow parallel processing. Allow all columns to be NULL so you can get the data in the staging tables or set the proper null conditions and have SSIS redirect these rows on error. Don't have any primary key / foreign key relationships in the staging tables, these can be validated in the stored procedure
Stored Procedures - Validate the staging data, any issues found would be added to the error log to be presented to the user or person performing the import. If there are no issues, import the data into the production tables. If there is existing data in the production tables, you could do a comparison and update if applicable.
How do I store data that is shared between databases?
Suppose a database for a contact management system. Each user is given a separate database. User can store his/her contacts' education information.
Currently there's a table called School in every database where the name of every school in the country is stored. School table is referenced as a FK by Contact table.
School table gets updated every year or so, as new schools get added or existing schools change names.
As the school information is common across all user databases, moving it into a separate common database seems to be a better idea. but when it's moved to a separate database, you can not create a FK constraint between School and Contact.
What is the best practice for this kind of situation?
(p.s. I'm using SQL Server if that is relevant)
Things to consider
Database is a unit of backup/restore.
It may not be possible to restore two databases to the same point in time.
Foreign keys are not supported across databases.
Hence, I would suggest managing the School -- and any other common table -- in one reference DB and then replicating those tables to other DBs.
Just straight out of the box, foreign key constraints aren't going to help you. You could look into replicating the individual schools table.
Based on the fact that you won't query tables with the SchoolID column very often I'll asume that inserts/updates to these tables will be really rare... In this case you could create a constraint on the table in which you need the FKs that checks for the existence of such SchoolID in the Schools table.
Note that every insert/update to the table with the SchoolID column will literally perform a query to another DB so, distance between databases, the way they connect to each other and many other factors may impact the performance of the insert/update statements.
Still, if they're on the same server and you have your indexes and primary keys all set up, the query should be fairly fast.
I need to create in my DB a log, that every action in the program should be written there.
I will also want to store additional data to it for example have the table and row the action was applied to.
In other words I want the log to be dynamic and should be able to refer to the other tables in the database.
The problem is, I don't know how to relate all the tables to this log.
Any ideas?
You have two choices here:
1) modify your program to add logging for every db access
2) add triggers to each table in your db to perform logging operations.
I don't recommend one logging table for all tables. You will have locking issues if you do that (every insert, update and delete in every table woudl have to hit this one, bad idea). Create a table for each table that you want to audit. There are lots of possible designs for the table, but they usually include some variant of old vlaue, new value, date changed, and user who did the change.
Then create triggers on each table to log the changes.
I know SQL Server 2008 also has a systemic way to set up auditing, this would be easier to set up than manual auditing and might be enough to lure your company into using 2008.